User Commands in Dyalog APL

Ray Cannon seems to have popularised this concept, which has its roots in APL*PLUS.  The idea being
The "user" part of the term is a little misleading - it means "people who are using APL" rather than "people who are using applications written in APL".
Ray has published several papers showing how he has developed similar functionality for Dyalog, but with Version 12.1 Dyalog have themselves added User Command functionality along with a "starter set" of commands.

What's important is to realise that User Commands are functionality written in APL, but which are not a part of the active workspace - they are held in separate files.

There's a Hard Way and an Easy Way

Dyalog's User Commands are implemented through the SALT code management functionality introduced a few years ago - their documentation tends to take the line that people will use SALT (and the Spice spinoff).  More independent souls may prefer to follow their own path, and this paper describes my own approach in a SALT-free environment.  It's not entirely SALT-free because the User Commands mechanism relies on some code which Dyalog have tucked away inside the #.⎕SE.SALTUtils namespace.  Either way, the "hard way" isn't actually that hard.

What You Need to Get Started

Obviously, you need Dyalog 12.1 or later.
You also need to make sure that you have the function #.⎕SE.UCMD and the namespace #.⎕SE.SALTUtils
You may see various warnings about needing to have SALT enabled - my experience is that so long as you have all the functions in all the right places you can do the User Commands thing with or without using SALT.

Where Do I Write/Store User Commands?

Last time I looked (it may have changed), Dyalog were being a bit "relative" in describing where User Commands are (or can be) stored.

What's Inside a User Command Script?

There are some requirements for the content of a User Command script, which make it compatible with the User Command mechanism - Dan Baronet has written a tool which is a part of the Dyalog product and makes this easier - but rugged individualists can go it alone.  The only warning I give is that if you get these bits wrong your script doesn't get recognised and you're into something of the dreaded edit/compile/fail/edit/... cycle.  The sensible thing to do is to develop the core functionality in your usual way and transfer code into script files - that way you only need to get stressed out by the housekeeping rules.

Best way is to illustrate by example, so what follows comes as a deconstructed User Command script.

First we have a little scene-setting...

:Class DogTools
    (⎕ml ⎕io)←1

Followed by the bits that Dyalog requires (this example was made by deconstructing Dyalog's supplied set).

AllCmds is just a list of commands
⍝ Spice Requirements -----------------------------------------------------------   

    AllCmds← 'displayr' 'filesearch' 'foldersearch' 'namespacelist'
    AllCmds,←'querymonitor' 'setmonitor' 'stoplist'

Help is what's going to appear when the programmer/user asks for help about a specific command, for example...

Command "displayr".
Script location: d:\dick\mycode\dyalog121\spice\dogtools

Arg: any array. Shows the array as per DISPLAY (with size hints). Assumes # by default

∇ r←Help Cmd;sw;t
      :Access Shared Public
       r←,⊂'Arg: any array. Shows the array as per DISPLAY (with size hints). Assumes # by default'
       r,←⊂'Arg: filename objectname. Is <object> in <file>'
       r,←⊂'Arg: foldername extension objectname. List of files containing <object>'
       r,←⊂'Arg: namespace name. Returns child namespace names'
       r,←⊂'Arg: function/operator name. Returns monitor results'
       r,←⊂'Arg: function/operator name. Sets monitoring for all lines'
       r,←⊂'Arg: namespace name. Returns location of all stop settings'

List is what's going to appear when the programmer/user wonders what User Commands are available, for example...

  ]?Dogon Research Tools
"??" for general help, "?CMD" for more specific info on command CMD

 Group                     Name               Description
 =====                     ====               ===========
 Dogon Research Tools      displayr           DISPLAY expression (with size hints)
                           filesearch         Does file contain object
                           foldersearch       List of files containing object
                           namespacelist      List of child namespaces
                           querymonitor       Query monitor
                           setmonitor         Set monitor
                           stoplist           List of stop settings

    ∇ r←List;i
      :Access Shared Public
      r.Group←⊂'Dogon Research Tools'
      r.Parse← ⊂''
      ((i←1)⌷r).Desc←'DISPLAY expression (with size hints)'
      ((i←i+1)⌷r).Desc←'Does file contain object'
      ((i←i+1)⌷r).Desc←'List of files containing object'
      ((i←i+1)⌷r).Desc←'List of child namespaces'
      ((i←i+1)⌷r).Desc←'Query monitor'
      ((i←i+1)⌷r).Desc←'Set monitor'
      ((i←i+1)⌷r).Desc←'List of stop settings'

Run makes it all happen...

    ∇ r←Run(Cmd Args);ref
      :Access Shared Public
      :Select Cmd
      :Case 'displayr'
          r←displayr ref⍎Args
      :Case 'filesearch'
          r←filesearch ref⍎Args
          :Case 'foldersearch'
          r←foldersearch ref⍎Args
      :Case 'namespacelist'
          r←namespacelist ref⍎Args
      :case 'querymonitor'
          r←QueryMonitor ref⍎Args
      :case 'setmonitor'
          r←SetMonitor ref⍎Args
      :Case 'stoplist'
          r←stoplist ref⍎Args

The code which is run is quite conventional, it's maybe convenient to use the same names for both command and function, but by no means compulsory...

⍝ Tool Definitions -------------------------------------------------------------

    displayr←{⎕IO ⎕ML←0                                                      ⍝ Boxed display of array (variant of Dyalog <display>)
          ⍺←1 ⋄ chars←⍺⊃'..''''|-' '┌┐└┘│─'                                  ⍝ ⍺: 0-clunky, 1-smooth.
          tl tr bl br vt hz←chars                                            ⍝ Top left, top right, ... 
          box←{                                                              ⍝ Box with type and axes.
              vrt hrz←(¯1+⍴⍵)⍴¨vt hz                                         ⍝ Vert. and horiz. lines.
              top←(1+⍴hrz)↑(⊃(¯1↑⍺)⌷hz,'0',⊂⍕¯1↑1⊃⍺),hrz                     ⍝ Upper border with axis.
              bot←(⍴top)↑(⊃2↓⍺),hrz                                          ⍝ Lower border with type.
              rgt←tr,vt,vrt,br                                               ⍝ Right side with corners.
              lax←(⊃¨(¯1↓3↓⍺)⌷¨(-1⌈¯1+⍴1⊃⍺)↑(⊂vt,'0'),¨⊂∘⍕¨¯1↓0,1⊃⍺),¨⊂vrt   ⍝ Left side(s) with axes,
              lax←(⊂1+⍴vrt)↑¨(lax~¨⊂' '),¨vt                                 ⍝ Pad and trim
              lft←⍉tl,(↑lax),bl                                              ⍝ ... and corners.
              lft,(top⍪⍵⍪bot),rgt                                            ⍝ Fully boxed array.
          deco←{⍺←type open ⍵ ⋄ (⍴⍴⍵),(⊂⍴⍵),⍺,axes ⍵}                        ⍝ Type and axes vector.
          axes←{(-2⌈⍴⍴⍵)↑1+×⍴⍵}                                              ⍝ Array axis types.
          open←{16::(1⌈⍴⍵)⍴⊂'[ref]' ⋄ (1⌈⍴⍵)⍴⍵}                              ⍝ Expose null axes.
          trim←{(~1 1⍷∧⌿⍵=' ')/⍵}                                            ⍝ Remove extra blank cols.
          type←{{(1=⍴⍵)⊃'+'⍵}∪,char¨⍵}                                       ⍝ Simple array type.
          char←{⍬≡⍴⍵:hz ⋄ (⊃⍵∊'¯',⎕D)⊃'#~'}∘⍕                                ⍝ Simple scalar type.
          qfmt←{(0=1↑0⍴⍵):' ',⎕FMT ⍵                                         ⍝ Pad numerics
              ⎕FMT ⍵}

          {                                                                  ⍝ Recursively box arrays:
              0=≡⍵:' '⍪(open ⎕FMT ⍵)⍪(⊃⍵ ⍵∊⎕AV)⊃' -'nbsp;                    ⍝ Simple scalar.
              1 ⍬≡(≡⍵)(⍴⍵):''(0 0)'∇' 0 0 box ⎕FMT ⍵                         ⍝ Object rep: ⎕OR.
              1=≡⍵:(deco ⍵)box open qfmt open ⍵                              ⍝ Simple array.
              ((⊂⍕≡⍵)deco ⍵)box trim ⎕FMT ∇¨open ⍵                           ⍝ Nested array.
 ∇ z←filesearch(file item);⎕IO;⎕ML;space;ntie;script;obnames;pos
 ⍝ Search for <item> in script <file>
      ⎕IO ⎕ML←0 3
      ntie←file ⎕NTIE 0
      script←⎕NREAD ntie 160(¯1+⎕NSIZE ntie)2
      script←{(~⍵∊⎕UCS 13)⊂⍵}script~⎕UCS 10
      ⎕NUNTIE ntie
      space.⎕FIX script
      pos←1+(↑script)⍳' '
      obnames←{⍵~' '}¨⊂[1]⍎'space.',((pos↓↑script)~' '),'.⎕nl 2 3 4'
   ∇ z←foldersearch(folder extension item);⎕IO;⎕ML;file;files
 ⍝ Search for <item> in script <file>
      ⎕IO ⎕ML←0 3
      files←↑¨('*.',extension) listfiles folder
      :for file :in (⊂folder),¨files
        z,←⊂(filesearch file item)/file
 ∇ z←namespacelist w;⎕IO;⎕ML
 ⍝ List of all namespaces
      ⎕IO ⎕ML←0 3
      z←((⊂w,'.'),¨(⊂[1](⍎w).⎕NL 9)~¨' ')~''
      :If ×⍴z
 ∇ z←stoplist w;⎕IO;⎕ML;nslist;stops;fnlist;stop;fn;ns
 ⍝ List of active stops
      ⎕IO ⎕ML←0 3
      nslist←(⊂w),flatten namespacelist w
      :For ns :In nslist
          fnlist←(⊂ns,'.'),¨(⊂[1](⍎ns).⎕NL 3 4)~¨' '
          :If ×⍴fnlist
              stops fnlist←(⊂∊×⍴¨stops)⌿¨stops fnlist
              :For (stop fn) :In (⊂¨stops),¨⊂¨fnlist
z,←(⊂fn),¨stop⊃¨⊂((1++\(⎕VR fn)∊⎕TC)⊂⎕VR fn)~¨⊂⎕TC

      SetMonitor←{⎕IO ⎕ML←0 3
          ~(⎕NC ⍵)∊3 4:⍵,' not a function or operator'
          (⍳↑⍴⎕CR ⍵)⎕MONITOR ⍵}

      QueryMonitor←{⎕IO ⎕ML←0 3
          0∊⍴⎕MONITOR ⍵:'No monitoring information for ',⍵
          ('Line' 'Times' 'CPU' 'Elapsed' '' 'Code'),[0](⎕MONITOR ⍵),⊂[1]⎕CR ⍵}

And, of course, there are some utilities used inside the script (not sure whether it's feasible to have User Command scripts depend on external utility code), and closing niceties...

⍝ Utilities --------------------------------------------------------------------

 ∇ z←flatten w;⎕IO;⎕ML;item
 ⍝ Flatten a nested structure
      ⎕IO ⎕ML←0 3
      :For item :In w
          :If 1=≡item
              z,←flatten item

      ⍝ File information for a folder
          ⎕IO ⎕ML ⎕USING←0 3(,⊂'System.IO')
          d←DirectoryInfo.New ⊂⍵
          fileinfo←d.GetFiles ⊂⍺
          info←⊃fileinfo.(Name Length LastWriteTime)


And finally...

I never really felt I'd been missing anything, probably because the APL*PLUS User Command functionality appeared after I'd substantively departed from the APL*PLUS world.

An opinion that's changed now that I've migrated a few development tools into the new (for Dyalog) paradigm - it's appreciably more convenient to have these tools automatically available rather than drag them into the working environment for every application.  The process of building and amending a User Command script is a bit laborious and fraught - but this is very much self-inflicted, Dyalog includes tools to help with the process.

But I think that Dyalog haven't yet realised the full potential of the concept (they've been very literal in following the APL*PLUS lead).  Some extensions I'd like to see explored...

Dan Baronet has shown that the User Command code can be run, for example...

   ⍴  ⎕se.UCMD 'displayr 2 2⍴3'
4 6

But that feels a little indirect/clunky.

What I'm wondering longterm - if User Commands truly became co-equal members of the APL world - is whether an evolution of this implementation might take us toward a situation where there could be an agreed tier of utility code available automatically, to finally put an end to everyone developing their own versions (and differently-named versions) of stuff like "delete trailing blanks" so that we might all think of ]dtb as commonly as we think of "reshape" (especially when we see that a lot of these things are already preloaded into the session file).

Page created 22 September 2009; Copyright © Dogon Research 2009