Wiki/Report of Meeting 2024-03-07

Report of Meeting 2024-03-07

Present: Art Anger, Ed Gottsman, Raul Miller, and Bob Therriault

Full transcripts of this meeting are now available below on this wiki page.

1) Ed has been working with ChatGPT 4 https://openai.com/gpt-4 and Ollama https://ollama.com/ to create Retrieval Augmented Generation (RAG) which is a way to ground then language capabilities of Large Language Models (LLMs). Interestingly, ChaptGPT could generate complex instructions to create a RAG in Olama. Unfortunately, this turned out to be an inaccurate hallucination. Ed also tried to use LLM's on J code and had limited success. He feels that someone with more knowledge of this may be needed or (and these are not mutually exclusive) J code may not be amenable to LLM augmentation. Ed and Raul have also been working with a third party to generate vectors that may create a smart cloud for the wiki information. Ed is dubious because the vectors may have 100 elements and reducing this down to a 2-dimensional map seems a bit of a stretch. Ed worked on browser search style interfaces professionally and has some experience in this area. Bob relayed that Marshall Lochbaum had told him that he thought that array language code would not work well with LLMs. Ed and Raul thought that at a gut level this was probably the case. Ed mentioned that APLCart https://aplcart.info/ has 6000 entries of code that may be able to be used this way because there are comments that may be searchable. Bob felt that APLCart also uses tags and that may be helpful as well. There is no corresponding application to APLCart, although Phrases might be extended.

2) Raul talked about the fix that he made to the 9!: page https://code.jsoftware.com/wiki/Vocabulary/Foreigns/9 with the CSS whitespace:nowrap to correct the issue that Jan-Pieter had seen in the page displayed on his Android phone. https://groups.google.com/a/jsoftware.com/g/forum/c/jNdtVMfsnoY/m/0hsw8PlAAAAJ Bob wondered if his version using CSS grid needed the same fix. Bob expressed concern that the CSS grid may not be used in all browser versions, but it also seems that most modern browsers will work with it.

3) Bob the showed his User Newcomers page that he had done as a prototype. https://code.jsoftware.com/wiki/User:Bob_Therriault/Newcomers At the time of the meeting only the top elements had been used in grid format. Since then the lower elements have been put into a grid as well which allows the coloured squares to be responsive as the page width changes. This is a very flexible approach and is preferred for the Newcomers page. For the other more standard category pages https://code.jsoftware.com/wiki/User:Bob_Therriault/Community there is not as much of an advantage for the grid display. The grid is better for editing information, but is a bit more difficult to set up so it remains to be seen if the tables form of display is preferred for simpler displays. There was also a discussion making the sidebar collapsible since it seems to take up a lot of real estate on small screens. Raul found an extension that should allow this functionality. https://www.mediawiki.org/wiki/Extension:CollapsibleVector The grid display simplicity is seen when compared to the table driven version of the Newcomers page. https://code.jsoftware.com/wiki/Category:Community_C With a bit more exploration of the grid system Bob will have a clear indication on the format to use for the category landing pages. Raul suggests that there may be more input from the users as the emulators seem to be limited in their effectiveness.

For access to previous meeting reports https://code.jsoftware.com/wiki/Wiki_Development If you would like to participate in the development of the J wiki please contact us on the J forum and we will get you an invitation to the next J wiki meeting held on Thursdays at 23:00 (UTC) Next meeting is March 14, 2024.

Transcript

Ah, so we'll let Ed start off, I guess, with what's new with the

JViewer and what's on the horizon.

Um, there's nothing much new with the viewer.

I spent directly, I spent some time with chat GPT and Olama trying to do what's

called retrieval augmented generation, in which you start with an existing

trained model of some sort, like a model trained by chat GPT, like GPT4.

And you, through a variety of transformations, train it on an

auxiliary dataset, which is usually text or PDF or something, texting.

Um, and it was interesting.

Um, first I asked chat GPT how to do it.

And it gave me very explicit instructions, including commands with options and

parameters for how to do it with Olama.

And I tried it and Olama wasn't, was rejecting the commands.

So I checked on the Discord, the chat GPT Discord, and I said, is this a

hallucination and the response was, yes, the command you've been given does not

exist in reality, it was made up by chat GPT and it was had three, maybe four

parameters with values, it looked very real, very plausible, I guess, plausible

is a word that's frequently used to describe chat GPT responses, but it was

entirely, and it was exactly what I wanted.

It figured that much out, but it was entirely fictional.

So I persevered.

I didn't ask chat GPT for help anymore on this particular topic.

And I did manage to trick.

Which version of chat GPT was it?

This was 4.

4.

Okay.

4.

Yeah.

Actually I tried it on 3.5 also, but in the end I wound up on 4.

And I actually trained up both a local OLAMA managed model, Mistraw, and I

trained up the chat GPT-4 using retrieval automated generations and ROD as it's

called, on the entire J wiki, which is about when you boil it all down, just

the text, you take away all the HTML formatting and so on, it's about 10

megabytes of material, and that took a while, but I did it.

And then I started to use the mean of a list of numbers, for example,

was always what I started with.

And at first it said, well, you just use the built-in function mean.

So it ran across mean enough times that it thought it was a built-in function.

So then I said, well, only using primitives, how would you find

the mean of a list of numbers?

And that had actually did pretty well on.

And then I would ask it, okay, how would you apply a verb to the

contents of each of a list of boxes?

And it didn't figure out under open.

Instead it said, well, you use the rank operator, I think is how it

characterized it, which is sort of plausible, but not right.

It's not right.

It said, well, you can apply the verb rank one to the list of boxes

and that isn't going to do it for you.

So, and I, there were others like that.

So my feeling is some, either somebody much more familiar with this domain

than I needs to dive into it and try to make it work or, and these are not

mutually exclusive, J is just not suitable for this domain.

That's entirely possible.

Or the entire GitHub source, J source repository needs to be fed into

retrieval augmented generation, retrieval augmented generation pipeline.

And then maybe it would start to figure out what the heck was going on.

That's entirely possible too.

I don't know what the answer is, but I, I am, I had sort of been looking for

something with which to augment the viewer.

I had been thinking in terms of some kind of retrieval mechanism, you know,

find me documents that have to do with X.

Haven't had any luck with that.

Haven't had any luck with that either.

So I'm at a bit of a standstill.

Now there is a gentleman who's been corresponding with role and with me,

who's got some background in, and search user experiences constructed using two

dimensional maps of document space, which in turn are built around things.

I don't fully understand.

So word to VEC and vector spaces and other things.

But I'm not really familiar with.

So I've been working with him and rule has been watching, trying to get to the

point where he, he, he is actually able to build SVG maps of large document

spaces that are clickable.

So he's, we're in the process in a somewhat halting fashion of trying to get

to the point where he's got the right kind of material to build a map of the

wiki where similar documents for some value of the term similar are nearby in

document space in this two dimensional document plane, I should say.

I'm a little dubious only because the vectors can have a similar

only because the vectors can have as many as a hundred or more elements in them.

And it seems to me that even using principal component analysis, getting it

down to two dimensions means that similar is a very slippery and vague concept,

but I'm certainly prepared to, to do the work necessary to make it possible for

him to produce such a map.

So we'll see what happens.

And that, that length is all I've got.

That's pretty good.

So the SVG maps he's talking about...

Nothing yet.

Well, yeah, but it's an interesting exploration.

And so the, the SVG maps that he's got, they're sort of like word

clouds for documents, is that accurate?

I think that's a fair statement for some value of the term word clouds.

The notion is that it's actually, it's something I worked on actually in my

professional career over the years on a strictly prototyping basis is browse and

search style user experiences where you might do a search and you see things light

up in document space, and then you look nearby, you look around hits and that

helps you find things that you're potentially interested in again, whether

it'll work with Jay is, is just an open question.

Yeah, I agree.

I think it should, it's best left as an open question.

Cause I've heard I think when I was talking on the podcast once with Marshall

Lachbaum and I was suggesting that maybe these patterns could be recognized and he

felt, no, there's no way he said, there's just, no, that's not going to, I don't

know why he felt that strongly.

Well, the patterns in the language.

So that if you've, you read the GitHub source into it, he's saying there's no

way it would know what was going on.

Yeah.

But that is my intuition too.

Um, but I'm certainly prepared to be wrong about that.

I've been wrong about AI for the last several years on a pretty consistent basis.

So I think that if, I mean, the basic thing that AI is working with is examples.

And when you're working with code and you're talking about examples, you have

to have example data as well.

It's just, it's not just sample expressions and, uh, and you need to test

it against actually evaluated.

You need to test how, what happens when you evaluate it until you can get the AI

trained to that point.

I think Marshall might be right.

It's just the same, have the same problem that people have of, you know, if you don't,

if you don't test your knowledge, you don't know, you can't find your mistakes.

Right.

What does this code do?

Even if I saw patterns, I wouldn't be able to tell me what this code does.

But once you hit, once you know, this code is similar to this other code, but I

couldn't, right.

Once you have example data and you can, you can say this thing produces as a

result, you can start saying this code is the same as that code because it does the

same thing in these cases or that.

The other thing is that Adam has, if memory serves some 6,000 commented sentences.

I don't, this is PL, but sentences is what we would call it in J.

Um, in the, uh, examples in the J documentation are J sentences that depend

on previous J sentences, so they use names that have been defined previously.

That's not so good.

Um, apple cart is as far as I understand it, independently defined sentences,

each of which can exist on its own with comments.

And I would be very interested if only we had such a thing for J with what a, an

LLM could do with that information.

But we don't, we don't have anything like that.

The only thing I, my experience with apple cart is it's not so much comments.

It's tags.

Like it's word recognition.

Oh, I thought it might be wrong.

Yeah.

Often.

Well, maybe there's a comment as well.

Often when he's, he's asking somebody who's looked for something and they can't

find it, he asked them what they were using as their search terms, and then he's

throwing tags in all the time based on that.

I see.

Um, but even those would be useful.

Have to be a problem.

Yeah.

That could work still, I think.

Um, but somebody other than I is going to have to tackle that.

I don't, I, I, APL is terra incognita for me and I'm feeling a little old in that

regard.

So you went to the newer language.

Yes, exactly.

Well, I made that decision long ago.

Yeah.

APL is kind of like standards.

I mean, there's, there's, I think then three dozen different implementations of

APL, maybe more by now.

And, uh, you gotta know, you gotta have a little comfort with that before you can

start tackling some of those issues.

There was one version of APL where quad IO was any floating point number, Honeywell

APL, I think it was, which made for interesting in every indices.

Yeah.

I know they have different model array models, like whether they're based or

whether they're floating or, or flat or flat, and that all the different versions

is there, there are definitely different flavors of APL.

And if you're using one with the idea of another, they really won't work well for

you.

So I mean,

Apple cart is dialogue, right?

I believe it's primarily, I believe so, but there's also a project that I heard

about recently and I have to go back a couple of episodes because I mentioned it,

but there's been somebody who's, uh, done across the different APLs.

And I think it, I think she's doing the work in the orchard, the APL orchard, but

which is another place discussion area.

But what she's done is she's taken all the different versions and basically done a

grid.

And so for a primitive show list, which ones and how they work within that, which

is almost like a Rosetta stone kind of an idea, not with great detail, but enough

to know if you're doing this here and you're doing it here, what, what are, what

are the similarities?

Do they correspond or not?

There are probably conclusions you can draw about a language that has gotten to

that point.

I don't speculate as to what those conclusions are, but I bet they're

significant.

Yeah.

Um, yeah, anyway, that's, um, it's certainly interesting to look at that

space.

Um, I mean, I think it's hilarious that you, you got chat GPT to basically troll

Olama giving you the wrong instruction.

There was a lot of intelligence.

There is nothing wrong with its desire to please.

I mean, it really wants to help.

There's no question about it.

Yeah.

The, the latest, I think it was, uh, dude, I forget the name, but there's another

model that just came out with a new verb.

It is attracting a certain amount of attention because

it was told, I'm sorry.

It was told about a pizza topping.

It was asserted that there was a pizza topping like Lego or something.

And its response was not to glitch, not to accept it, but to say, I think maybe

you're trying to see whether I'm paying attention.

Wow.

Which is not a bad response from a putatively unintelligent thing, entity,

whatever the right term is.

Um, so they really are getting really, really good.

And, but the problem is, um, they're getting sufficiently good that if you

think about a programming shop, you've got programmers, you've got people you've

got, maybe your programmers write the unit tests, so maybe you have other people

who write unit tests for you, and then you have QA and QA is doing automatic

and manual testing on the final application.

If you can talk to a code generating LLM, you can generate a system in

potentially a few seconds and leave it up to QA and maybe the unit testers,

although probably not, but certainly QA to decide whether you've got a working

thing or not, and all of the concerns that we've developed over the years

around reusability, modularity, documentate, excuse me, comments.

Um, abstraction, those are all sort of artifacts of the need for human

beings to understand the code.

And if human beings don't need to understand the code, if they just need

to test the code, then we don't need programmers anymore.

Well, that's worth thinking about some of the issues that can show up in

testing, like edge cases and things like that.

Um, yeah.

So it's, it's interesting.

The, at some point you need experts in the, in the field to do, to do the

training and you also at some point later on need experts in the field to

sort out problems that arise because it's been drifting off course because

over, over ambitious sales pitches and, and the misunderstandings that creep

in, you, you, there's still an ongoing need for, for parameters.

It's just not necessarily the same role.

And it's true.

It's, and it's not clear that you need as many programmers as you needed before.

But the other thing is we also need non-programmers in areas

that we've been abandoning.

Like we, we don't have very much manufacturing nowadays and you need

people with expertise in manufacturing to take, to advance manufacturing into

the, into the new areas that programming could support and we don't have the

people and so that's been languishing.

And other countries have been, been, that have been doing manufacturing

are catching up with us and exceeding our capabilities in some respects

because of that lack of personnel.

So that's, that's, uh, you know, you're talking about an issue that's, that's

rather subtle and, and non-intuitive in, in, in the implications.

Yeah.

I, I was struck by some recent work and by recent, I mean, apparently

the last couple of weeks, things are moving quite rapidly where the knock on

AI, and this is indirectly related, the knock on AI generated images and

increasingly video, which is getting quite good, uh, is well, you know, the

characters don't have thumbs or they have claws or whatever, and then what do you do?

Or butter grumble.

I don't know.

Well, there's work being done on feedback mechanisms where you can freeze most of

the image and just point out the thumbs or the claws and give a new prompt, a new

request that says, all right, hold everything else steady and just fix the

thumbs, fix the claws, they should be real thumbs or they should be fingers.

And that seems to work quite well.

I don't know what the equivalent is in a system that's been generated by the, by

the AI, but it strikes me that there might be some mechanism where you could

say, I love this, but there are the following edge cases that you haven't

considered.

And the thing to think about there also is media that's not images like, um,

no architectural drawings, um, you know, mechanical, mechanical designs,

computer circuit, and there's a whole bunch of other areas and, you know, the

focus on that one media doesn't, it's, it gets into this whole expertise thing.

You got to have people that are interested and motivated and working in

that area before you get activity moving, moving the field forward.

We've got a lot of people interested in images.

We don't have so many people interested in machine design.

Right.

But it'll come.

It'll come.

Maybe not before some other country eats our lunch, but it will come.

Well, we have, we have, sorry.

It's easier to catch up than it is to advance, to be a leader, I guess.

That's true.

That's also true.

That's true.

Uh, and there are problems with being a leader.

You tend to standardize on things early and the rest of the world learns from

your mistakes and actually leapfrogs you.

So Bob, we've drifted a bit and I apologize that that is really all I've got.

So if you wanted to move on to the next agenda item, which I think

was CSS, that would be fine.

Yeah.

Um, and, and, uh, before I dive into the CSS stuff I've done, I haven't heard

back from, uh, John Peter yet.

I should actually just check my email to see whether he has replied.

Oh, maybe he has.

Or is this you replying at all?

Oh, okay.

So maybe, or I'll film me on what you did that you think fixed it.

Cause I haven't heard anything back from John Peter yet.

I just go look at the page and the, the history log.

I just, I, I did a, um, I think it was white space, no wrap.

Um, attribute in CSS and I think that fixed it.

Okay.

So what was happening was the, when you had the multiple divs,

it was wrapping on themselves.

Yep.

Huh?

Because it was a small, very small window.

Right.

I don't know why.

You know, they, they decided text should wrap and not push down, but

that's the way it was working.

So.

Oh, okay.

Cause, um, I'd sent an email asking him about the alternative way that I'd done it.

I haven't gone back and changed yours yet, but the alternative way to see

whether that made a difference.

And my guess is if it is wrapping, that might've made a difference because

I'm running mine horizontally rather than stacking them.

Right.

So it's possible that could make a difference.

I was basically looked up what people have been talking about with the, with the text

over with the text overlap issue, and that was the simplest handling that there was a

number of different handlings for it.

And that was the simplest one that I saw.

So I went with that one.

Okay.

Well, that's interesting.

Well, that might be something when he gets back to find out whether the

version I was doing is different.

In which case that might be a preferred style or whether we should be including

no wrap with all the CSS to make sure that it's covered the concern I had was

when I was looking back on, when can I use it said Android, uh, Chrome, Android

one 22 supported just about everything.

So my question back to him is what are you using just to find out

whether it's an older version?

Um, but it would be unfortunate if it turned out that he was using a common

version that didn't support because he's using a Samsung, I think he said.

Yeah, but it's the version of, uh, Chrome Android that he's using.

Right.

Oh, I didn't look to see which version he didn't say what version.

That's why when I replied, I asked him.

Cause I was wondering if he's using something earlier than one 22, and it

turns out that a lot of people are doing that, even though CSS grid's been in for

five years, actually more than five years, probably closer to six or seven years now.

Um, even though it's been around for that long, if, if there's a lot of people

who still aren't able to use it, then it becomes a tool as useful as it is.

And it seems to me to be very useful.

Um, we probably have to watch how we're using it or end up writing, you

know, version dependent stuff, which is really something that I had

really hoped we might avoid.

But you never know, I guess sometimes you just have to bite the bullet and do it.

Seven years is a long time.

It seems that people be at least that, uh, up to date.

Yeah.

Well, yeah, I just, I, I hate to draw that line, um, for people, you know, and say,

well, no, you can't use this wiki because your stuff's too old.

Well, there were other approaches to fixing it and I could throw in some of

the additional handlings too, if it turns out that it's not working.

If it turns out that this one didn't fix it.

Yeah.

Um, there's, there was a one that they all had to do with, well, I don't

know quite what the commonality was.

It was one that had to do with white space, putting a white space prefix on

things, and there was another one that didn't even look at too closely.

To show what I've been doing with, um, this, I took my newcomers page

and I didn't do all of it yet.

I'm planning to, this is back in my, uh, Oh no, this is on the wiki category.

Maybe I should flip back to my user.

Here we go.

This is my newcomers page.

Hopefully we're seeing what I expect to see here.

Let's see.

Tell quickly.

Yep.

You're seeing what I'm seeing.

So that's good.

Um, jumped over here and I want to be back.

What's going on now?

Come on.

What are you doing to me now?

Okay.

Let's try going back a page then.

Oh goodness.

Made a big botch of this.

I am going to change this.

This thing is in my way.

Okay.

Good.

Um, this is my new newcomers page and what I, and I'm just using it within my user.

Area.

So it's really a sandbox.

Um, but what I've done is I've taken these top four items.

So the, uh, the table of contents, the video group, the tree group, and this box.

And what I've done is I've actually put them in a grid.

And what that gets me is when I do something like resizing, I can have

stuff like that happen where it, it actually stays stable.

I don't have things moving, uh, out of relation to each other.

Now there's a few things I've got to clean up.

I've got to put some padding around this in here, but that's pretty easy to do.

Um, and I'm looking to actually do the rest of these because what I can do as

well with CSS is I can have this, um, display say up to five or six boxes in a

row, and then it'll just feed down to the next box.

So as I get narrower, I haven't done this yet, but rather than having

just the boxes get narrower.

What'll happen is I'll actually go down to three boxes here and then two boxes.

And I think two boxes is as narrow as I've got.

The big advantage is if, when I look at this, yeah.

I'm not sure we're seeing what you're seeing.

Okay.

So nothing is moving.

You resized and we saw nothing change.

Okay.

So that's interesting.

Yeah.

Okay.

Um,

yeah, you're right.

I, that's interesting.

So I guess what Zoom's doing is it's, it's, it's saying you can do

whatever you want with this window.

I'm just locking on what you presented me.

I don't know.

Cause I'm, I can see what I'm screen sharing and you're right.

I changed the, the, the width of it and it doesn't change the screen sharing at all.

Yeah.

Um, well, um, can you show your whole desktop?

Yeah.

I can go back to that.

The.

Your screen.

Desktop.

See if that works.

So there you go.

Although at a certain point it disappears.

That's interesting.

Oh, well, I guess it just fades out at a certain point.

Anyway, that's what's happening.

But what you can see is these areas here, this stays centered with regard to these two.

So I don't have this trying to recenter itself.

It's actually on a grid and it's taking this full width here and it's squeezing that down

independently of what this is, but it keeps these really close together.

So it's not squeezing that down independently of what this is, but it keeps these relative

positions all the same.

And my goal is when I get down to this area is to make it a grid because I can change.

The number of boxes that are shown automatically as it gets narrower.

And when I've gone back to like looking at this page on my phone, it automatically comes

to something like this, it's narrower.

So I don't have to worry about something not showing up, which I do when I go back to the

other versions of, let's say, I think this is this.

There's the old version.

So you see this sort of thing, it being cut off at that width.

Yeah.

And it also doesn't line this up.

If this is shorter than this one, doesn't take this into account.

There's all sorts of things that start happening where the grid really cleans up the

situation.

So I don't have to worry about things being cut off.

I can just make it a grid.