Living on a LowSALT Diet

Dick Bowman
3 November 2006 (updated 30 November 2006, 29 January 2007)

The work described here is an update to the approach described in the earlier "Storing Class Definitions on File" and it may be useful to read both in conjunction.

Note that it is a report on work-in-progress and that there may have been as-yet undocumented changes and enhancements since the time of writing.  Please feel welcome to contact the author.

Objectives

As before, the goal is to share utility code across a number of applications which are in ongoing development so that changes made while working on one application percolate automatically to others.
A secondary objective was to learn more about some of the new features of Dyalog APL/W Version 11.
The work shares SALT's goal of holding APL code in non-workspace (and effectively non-APL) formats.
Although others have adopted a more extreme approach to code maintenance and put everything into their own SALT files, at this time I am following a hybrid path with utilities held in script files, but the application specialities as workspace code.
I'm not a convert to the "you can edit APL code using your favourite text editor" camp - partly because I don't have a favourite text editor, but mostly because I want to edit using a toll which is sensitive to APL syntax and lets me go off on side-tracks using APL as a tool.  I want to edit my APL where it's supposed to live and breathe.

Why LowSALT

Dyalog themselves include a code maintenance system within the Version 11 product, and this may well prove to be the most satisfactory tool for serious commercial code production (especially as Dyalog have to stand behind it).  But it is provided as a layer above the "barest bones" and may not provide a complete match between the functionality required and the functionality offered.

The code shown here starts at a lower level and allows me to take it whichever way I want - the code is available and others are welcome to bend it to their own needs.

A word (or two) of warning.  This code (like SALT itself) is built around undocumented features of APL/W Version 11 and may not work with future releases of APL/W.  You have been warned (although if it stops working, the author is also in something of a pickle).

Bootstrapping Utilities into an Application

Here's the code inside the latent expression of an application workspace

 ⎕IO ⎕ML ⎕PATH←0 3 '↑ '
 umode←#.GetEnvironment'umode'
 utils←'BLANKS' 'CONFIG' 'DOGON' 'FILE' 'GEN' 'GUI' 'HELP' 'HTML' 'STRINGS' 'TODO' 'WHOLES'
 utils,←(umode='d')⌿⊂'TOOLS'
 utils,←'DogForm' 'DogMenu' 'DogPopup' 'DogTool'
 utils,←⊂'DogComponentFile'
 utils←umode #.⎕SE.UtilLoader.UtilLoad utils
 umode #.⎕SE.UtilLoader.UtilBoot''
 #.APPSPACE.APPMain'' 
 

UtilLoad checks for dependencies, puts the code where it belongs and amends ⎕path (opinions differ, I prefer to elide full namespace qualification in my code - computers can do the dull things like find out where things are, and it lets me move stuff around more readily).

 z←umode UtilLoad w;⎕IO;⎕ML;util;name;space;subfolder;junk;source;ufile;utilinfo;utillist
      ⍝ Utility loader
 ⎕IO ⎕ML←0 3
 utilinfo←UtilInfo''
 z←''
 utillist←utilinfo UtilDepend w
 :If umode∊'dD'
     :For util :In utillist
         :If (⊂util)∊0⌷[1]utilinfo
             name space subfolder←3↑((0⌷[1]utilinfo)⍳⊂util)⌷[0]utilinfo
             ⎕PATH,←' ',space
             umode UTellem'UtilLoader: Loading <',util,'>'
             ufile←⎕NEW #.⎕SE.UnicodeFile((UtilFolder''),subfolder,name,'.',ScriptExtension)
             source←'load'UtilStamp ufile.NestedText
             ufile.NestedText←source
             ⎕EX space
             space ⎕NS''
             junk←#.UTIL.⎕FIX source
             ⎕EX'ufile'
             z,←⊂util
         :Else
             ∘ ⍝ Unrecognised space, UtilInfo needs to be extended
         :EndIf
     :EndFor
 :EndIf


UtilDepend fills out the utility list where one (or more) of the explicitly-named utilities depends on something else (at present I'm not using the :Include keyword, which may or may not help or hinder here).  It's essentially mundane - code not shown.
UTellem is also quite dull, using the case of umode to determine whether the right argument should be shown in the session.

UtilBoot is pirated from SALT, and sets up the necessary changes to make namespace script edits automatically saved (this is the "undocumented callback" stuff).
  ∇ UtilBoot w;⎕IO;⎕ML
     ⍝ Utility bootstrapping (adapted from SALT)
      ⎕IO ⎕ML←0 3
      ⎕SE.⎕WX←1
      UtilEditor←⎕SE.Editor
      UtilEditor.onFix←'UtilFix'
      '⎕se'⎕WS'Event' 9998 'UtilFix'
    ∇

Commenting Conventions

Several documentary/administrative comments are included in the script files and automatically amended when appropriate:

Type/descriptionExample
Dogon - signalling that this is "mine" and therefore subject to saving on edit⍝:Dogon Research
Created - time/user/workspace of creation⍝:Created ⋄ 2007 1 25 12 2 14 88 ⋄ dick ⋄ .\htmltools
Loaded - time/user/workspace of last load⍝:Loaded ⋄ 2007 1 29 14 2 12 828 ⋄ dick ⋄ .\htmltools
Saved - time/user/workspace of last save⍝:Saved ⋄ 2007 1 25 14 15 56 625 ⋄ dick ⋄ .\webstats

Utility Information

There's a UtilInfo that describes where utilities are to be found (relative to a single parent folder) and where they should be put by the loader.  It returns a five-column matrix (maybe it should be defined as a matrix - I'm undecided, as yet, whether the utility loader code should itself be a script).

NameTarget NamespaceSubfolderCommentDependencies
What's the utility called?Where should it be put in an application workspace?Is it in a subfolder of the main utilities folder?  The assumption is that all locally-generated script files will be held within a single folder tree.Not presently usedA list of utility spaces which are needed to make this one work

So, in part...

'REPORTS' '#.UTIL.REPORTS' '' ''(⊂'PRINT')
'SQL' '#.UTIL.SQL' '' '' ''
'STRINGS' '#.UTIL.STRINGS' '' '' ''
'TODO' '#.UTIL.TODO' '' ''('BLANKS' 'REPORTS' 'DogPopup')


Note that dependencies are resolved recursively.

On-the-fly Utility Maintenance

With the application running we can interrupt (or have it fail) - code changes are detected by the editor callback, with UtilFix leaping into action...

 UtilFix w;⎕IO;⎕ML;junk;umode
     ⍝ Utility fixer - based on Dyalog SALT
 ⎕IO ⎕ML←0 3
 umode←#.GetEnvironment'umode'
 :Select 1⊃w
 :Case 'Fix'
     umode UTellem'UtilLoader: UtilFix'(1⊃w)(4⊃w)
     :If '⍝:Dogon'{(⊂⍺)∊(⊂⍴⍺)↑¨⍵}2⊃w
         umode UTellem'Enqueuing...'
         ⎕NQ'⎕SE' 9998,2↓w
     :EndIf
 :Case 9998
     umode UTellem'UtilLoader: UtilFix'(1⊃w)(4⊃w)
     junk←(#.GetEnvironment'umode')UtilSave,4⌷w
 :EndSelect


Notice that it is run twice, first with an argument of Fix as part of the "normal" editor fix process, then a second time with argument 9998 if what's being fixed is on the #.UTIL.loaded list.

Having set the callback in the latent expression the mechanism stays in place even after the application stops running, so I have the choice of editing on-the-fly or as a separate task.  There are a couple of areas needing further attention/investigation.
Although I have a discrete "class maintenance" application I'm not using it at present.  Working on-the-fly with the tools described here seems to serve my current purposes well enough.

Searching

A perennial problem - writing an application, you know that there's a utility that does what you want, you know what it's called, but you don't know where it is.  UtilSearch takes care, with an argument which is either just a name or a nameclass and name (so you can look for "a function called Wally")...
z←a UtilSearch w;⎕IO;⎕ML;nc;name;utilfolder;utilinfo;util;folder;tempspace;source;ufile;junk
⍝ Search in utilities
 ⎕IO ⎕ML←0 3
 z←''
 :Select ≡w
 :Case 1
     nc name←0 w
 :Case 2
     nc name←w
 :Else
     ∘ ⍝ What are you fooling around at?
 :EndSelect
 (tempspace←'#.Temp',(⍕⎕TS)~' ')⎕NS''
 utilinfo←⊂[1]1 0 1 0 0/UtilInfo''
 :For util folder :In utilinfo
     a UTellem'Looking at ',util
     ⎕EX tempspace
     tempspace ⎕NS''
     ufile←⎕NEW #.⎕SE.UnicodeFile((UtilFolder''),folder,util,'.',ScriptExtension)
     source←ufile.NestedText
     ⍎'junk←',tempspace,'.⎕fix source'
     :Select nc
     :Case 0
         z,←⊂(0≠⎕NC tempspace,'.',util,'.',name)/util
     :Else
         z,←⊂(nc=⎕NC tempspace,'.',util,'.',name)/util
     :EndSelect
 :EndFor
 ⎕EX tempspace
 z←z~⊂''

Other Stuff

UtilNew is used to make a new script (either a Class or a Namespace - at present I'm not up to Interface)

 z←type UtilNew(name location subfolder comment dependency);⎕IO;⎕ML;filename;utilinfo;⎕USING;ufile;template
⍝ Create a new utility
 ⎕IO ⎕ML ⎕USING←0 3('System' 'System.IO')
 filename←(UtilFolder''),subfolder,name,'.',ScriptExtension
 :If ~File.Exists filename
     utilinfo←⊂[1]⎕CR'UtilInfo'
     ⎕FX⊃utilinfo,⊂'z,[0]←',∊'''',¨name location subfolder comment dependency,¨⊂''' '
     location ⎕NS''
     template←⊂'('':',type,' '',name)'
     template,←⊂'(''⍝:Dogon Research'')'
     template,←⊂'(''⍝:Created ⋄ '',(⍕⎕TS),'' ⋄ '',Environment.UserName,'' ⋄ '',⎕WSID)'
     template,←'('':End',type,''')'
     ⍎location,'←0 ⎕fix ',∊template
     ufile←⎕NEW #.⎕SE.UnicodeFile filename
     ⍎'ufile.NestedText←⎕src ',location
     (#.GetEnvironment'umode')'Utility ',name,' created, session file )SAVEd'
     ⎕NQ'⎕se' 'FileWrite'
     z←1
 :Else
     z←0
 :EndIf


Everything else is fairly mundane and as described in earlier papers referenced above - a file containing all the code can be obtained from Dogon Research (on the usual no-fee/no-support basis)

And Finally...

It's still in a state of flux, but in daily use and seems to be approaching functional/reliable stability.  More changes are likely to happen, and should be documented in a later version of this document.

Acknowledgement

The code shown here was developed (at least in part) by reverse-engineering Morten Kromberg's SALT - and could not have been written without Morten's pioneering development of SALT.

Copyright © Dogon Research, 2006-2007.  Material from the Dogon Research Music and APL Pages may be freely redistributed, but copyright is retained and attribution to Dogon Research is requested.