Introduction | Installation | Using WinHelm to Analyze and Amend Data | Creating a New Application | Issues | Under The Lid | Where Does APL Fit In? | Summary | Conclusions | CodeWork Responds
WinHelm is an OLAP package written in APL and developed over many years by CodeWork of Italy; long a DOS product, it evolved into a Windows (3.1) application a year or so ago and has most recently been upgraded for use with all Windows variants. There are both 16-bit (for Windows 3.1 and 95) and 32-bit (for Windows 98 and NT) versions. This review is of the freely-downloadable 16-bit version (version 1.1) installed onto a dual-processor Pentium II machine with 128Mb RAM running NT Workstation 4.0.
For those not completely clued-up on these things, OLAP is a term used to described multi-dimensional data analysis technology often (but not exclusively) associated with data warehouses; classically handling the division/brand/product/period type of breakdown of a company's sales figures. Many examples abound in corporate-software-land (PC Express is a long-standing example, now rebranded Oracle Express; EssBase is another). Traditionally a high-tariff software market, it may be due for a significant shakeup since Microsoft have integrated their recently-acquired Plato product into the latest version of SQL Server.
The APL enthusiast will have seized on the word "multi-dimensional" and realizing that the essence of OLAP is selection and aggregation seen that the application area is a natural for APL; indeed anyone (with the right background) who carries out an appraisal of OLAP tools quickly realizes that what they most lack is a natural grammar for expressing multi-dimensional operations. Indeed, you could say that the most (technically) successful OLAP packages are those where the developers are furthest down the road of re-inventing APL - this particularly applies when you realize that the natural data evolution is from the single hypercube to multiple hypercubes. What could be more appealing, then, than to look at an OLAP package authored in a most appropriate language?
Return to top
The 16-bit version was downloaded from CodeWork's Web site (http://www.inrete.it/cdwk/eng/homee.html will get you there), along with documentation; there is about 6Mb to download (including a hefty help file and 123-page Postscript manual - also available in .pdf format) and transfer speeds are variable. Once unzipped the installation process is speedy, but a little too automatic for my liking (it insisted on installing to c:\codework rather than allowing me to select a destination - it resisted subsequent attempts to relocate it). The version under review was written in Dyalog APL (version 7.1) The opening screen looks like this...
Clear (enough), but idiosyncratic. It might well be justifiable to be unhappy about Microsoft rules and regulations and CodeWork may well have good reasons for laying out things as they do (despite the excitable babbling of the press, many users only ever use one application - What They See Is What They Like) but I would have felt happier to at least see a Help option on the main menu. Anyway, here's an opportunity to read the documentation...
Return to top
Selecting the WorkSession button opens the next dialogue...
After selecting a database the most common operations are data analysis and data entry. Three sample databases are supplied - all three-dimensional (MSALES - 60 60 70 elements, 2megabytes; PROFLOSS - 150 10 30 elements, 300kilobytes; YSALES - 60 5 70 elements, 175 kilobytes). We will use YSALES for this exploration of WinHelm. Press the Data Analysis button...
There are many options here for selecting and rearranging the data; once we have chosen the view we want we can open it for browsing...
Equally easy (backtracking to the WorkSession form) to open a view for data entry...
You get the general flavour; the interface is a bit idiosyncratic, but it allows the user to move around and do "most" things quite easily. I think it would definitely be an advantage if (at least) help was available from a menu item - although in practice I often prefer to have a paper manual with me, because it's easier to view both the "problem" and the "solution" simultaneously.
Return to top
New databases are created as part of the administration function, available from the worksession form; the key dialogue being
It's worth noting that WinHelm offers some pre-formulated dimensions (for example, rather than choosing to set up the Month dimension from scratch I could have used the predefined MONTHS namesystem).
Notice also the element counts; current element count allowing WinHelm to pre-build the database - so that as data is entered it can be placed into pre-allocated locations (the file has been created 971Kb in size) and the maximum element counts. It's not really clear what function the maximum counts serve.
Now that we have the new database data entry and analysis can proceed.
Return to top
After working for a while with the supplied examples and having begun to set up new databases a number of issues begin to assume prominence...
This sort of application does not live in a vacuum; typically it will be required to suck data in from other sources and to supply data to other applications.
Sending data out of WinHelm is fairly straightforward, with two main choices.
Populating a database from a flat data file is fairly straightforward, once you've done it the first time. The rather idiosyncratic dialogs guide you through the process; for example specifying field location...
It is fairly frustrating to find that every step has to be repeated from scratch when (as inevitably you do, with all software) you make a mistake setting the process up. I did not look in great detail at what are called "procedures" and which correspond to macros or scripts in other applications. Basically, the user's actions while using WinHelm are recorded during the session and may be saved for later editing and reuse. I would expect that procedures would be fairly extensively used in a production scenario.
So far, WinHelm can be seen to offer quite adequate facilities for recording and analysing single data values identified within the three-dimensional structure. What about needing to hold multiple values? For example we might be recording quantity, price and discount level. The answer seems to be, not directly. Each set of data values needs to be held in a separate database; the user may then combine data from the two (or more) databases using a secondary cube. What this does is to arrange the various data values along a single dimension.
I'm not really convinced by this; I think it's the first hint that WinHelm isn't really offering a fully-fledged industrial strength multidimensional application. People's views may differ on this, and let's be quite clear that WinHelm is an order of magnitude more suitable for three-dimensional data than (say) Microsoft Excel. But it hasn't got me where I want to go yet.
What about the situation of needing to handle more than three dimensions? Again, it's not hard to come up with real-life situations such as the manufacturing company which wants to track sales by product, factory, sales office, customer, month. You can do it in WinHelm, but by subterfuge.
There is no way to escape the intrinsic three-dimensionality which underlies the product. To hold data with higher dimensions the user has to devise a mapping scheme for combining logical dimensions to fit the physical database layout. WinHelm provides many tools to help in this process, not only making it relatively painless but also allowing for the inevitable changes to design which are going to happen in practice. But, it's a three-dimensional tool - at heart.
Return to top
The evaluated version was written in Dyalog 7.1; the workspace contains about 600 defined functions (and no defined operators) which are mostly locked. I made no attempt at reverse-engineering anything but did take a look at the unlocked functions (which were clear enough); I don't know whether paying customers are either allowed or encouraged to modify the code (potentially an advantage for the customer, but a nightmare for the vendor). There do not appear to be any function files (the workspace is it).
While running the application I experienced a number of GPFs; but I would tend to attribute these more to the perils of running a 16-bit application under Windows NT than to either CodeWork or Dyadic Systems.
Return to top
From the viewpoint of the end user APL is essentially invisible; this is a package which happens to have been written in APL - no more, no less. Although there are computational features which have that APL feel they are all made visible to the user as pre-packaged options.
An area where WinHelm benefits from its origins is the naturalness of cube operations - this is something which OLAP packages only really handle elegantly when the package author is at home in a multidimensional world. It is definitely an aspect in which WinHelm is superior to many alternative packages.
Return to top
I think we need to look at WinHelm in two quite distinct modes
WinHelm occupies a middle ground between the two-dimensional world served by the likes of Excel or Access and the full dimensionality of the heavyweight OLAP applications. I've seen far too many people struggling to take spreadsheet packages where they don't fit (I never saw Lotus Improv - or whatever it was called - in action, which I sort of regret); my guess is that WinHelm would make life easier for very many of these people. The problems for it to overcome is that it's from "outside" and corporate solution-buyers seem averse to even looking at tools from outside the supermarket (or favoured suppliers); the idiosyncratic user interface doesn't help in this regard.
It would be good to see WinHelm get a cosmetic makeover; to play the old trick of "it looks like it's part of Microsoft Office, so they'll probably let me use it". I'm no fan of conformist look-and-feel, and CodeWork is probably intellectually correct in having an interface which is appropriate to the job in hand. But little things like not being able to reach Help directly from a menu probably aren't going to help mainstream user acceptance.
If you have a three-dimensional problem WinHelm is probably a good choice of solution, and it's probably a better solution than (at least some of) the higher-dimension packages which are available to the deeply-pocketed corporate buyer.
Multidimensional data applications are a natural for APL; there are commercially successful packages which have no relationship to APL and it shows (for example, look at the convolutions which are necessary in EssBase if you want to perform any computation across the data cube). Indeed it's easy to assess the capabilities of an OLAP package by seeing how close the vendor is to reinventing APL.
What's disappointing about WinHelm from an APL perspective is the way in which it doesn't naturally progress beyond the third dimension; it goes there, but it's hard work. The problem, of course, is that a true APLer doesn't see the complexity of a multidimensional world in the same way that the rest of the population does - what's a natural extension for us is hard for many to visualise.
The key to a successful OLAP application is having a file structure which maps the data cube efficiently; do the sums and you soon see that the implied data space of a multi-dimensional cube is far larger than fits easily into "normal" data files. It may be the J's recent adoption of memory-mapped files could be a significant step in terms of making these problems go away but, at least for now, the translation between a linear file space and a multidimensional logical data shape (or indeed multiple higher-dimension data shapes) is the key to successful OLAP. WinHelm seems to do well within its own limitations.
The question which WinHelm begs for me is the Open Source conundrum. We've seen that at least for Linux an open source code policy combined with large numbers of enthusiastic programmers has built a "product" with widespread acceptance. I don't know whether this has been achieved in any other domain. But WinHelm makes me wonder whether something similar could happen for APL with OLAP applications.
The commercially widespread packages are deeply technically flawed; WinHelm offers a solution to the data mapping question and a framework for user presentation. What would happen (assuming CodeWork could put arrangements into place which protected their commercial interests) if the whole APL world could be let loose on making WinHelm2?
What would be the commercial deal for CodeWork and the programmers? Consultancy, consultancy, consultancy. No matter how effective the package, the skill of visualising multi-dimensional data and data transformation is a rare one; it's a skill which has osmosed into the world of APL. A good APL programmer does this stuff naturally, most people don't. Multidimensional analysis is the life blood of the business back office; they need to do it now with single hypercubes and they need to do it tomorrow with multiple hypercubes - and the (user) tools don't exist. How to organise the data, how to acquire the data, how to analyse the data, how to present the data. APL is a raw tool which can do this, but we also need to construct tools at a higher level rather than just throw in APL or build the lower tools over and over again. WinHelm as Open Source might take us there.
Return to top
This has been a hard review to write; WinHelm looks like an interesting application but seems to be quite limited in its aspirations and achievements - I would like it to have gone further. But robustness and stability also have their attractions. It's a nice example of a commercially successful APL application that does what it does effectively and reliably.
My general impression is that WinHelm is easy to use and does a good job on its own terms; multi-dimensional data is hard for many end users to come to terms with and they need tools like this. I would expect that paying customers receive quite a lot of assistance and support from Codework as they deploy the product; certainly my impression is that the knowledgeable product consultant would take WinHelm effortlessly beyond the places where I (from a novice perspective) see limitations creeping in.
For me, what's probably more important is what WinHelm offers as potential for the future. If we were able to take a package like this, which moves APL into a (fairly generalised) business area and have many more programmers swarming over it than a single commercial enterprise could support we might just be able to move out of repeating ourselves and toward the sort of success that we see for Linux. Remember, the money is in consultancy - not just in selling product.
Return to top
Thank you for the time you are spending on WinHelm.
We read you review with keen interest and realized that your task was made unduly hard by lack of support on our part (RJB notes that no advice or support was sought during the preparation of this review).
A few points:
Platforms. You would have liked the DOS version much more. This is still the reference version for all future work and is more complete, reliable and finished. Also, access to APL is a keystroke away from HELM use. We sold one DOS-HELM to a bank a few days ago, and expect to sell more in
the future, in cases where processing is regarded as a batch operation.
Reliability of Windows versions. We consider the 16-bit version unreliable under Win 3.1 (due to the compound effects of Win 3.1, Dyalog 7.1 and WinHelm code), acceptable under Win 95/98 and UNTESTED with Win NT. The 32 bit version is ok with both Win 95/98 and Win NT (from a GPF viewpoint).
GUI solution. We are not proud of it, but our customers find it bare-bones but still usable and productive.
You are quite right in saying that Codework disregards Microsoft's recommandations on GUI style and conventions. In due course, we plan to restart from the DOS version and redo the user interface in HTML, XML or something else that sets us free from Microsoft's rule.
3 dimensions. Initially, this was a straight jacket and a minimal choice (you cannot have less than three dims to accommodate variables, periods and items and give these three axes specific treatment). But with the breakthrough of the Crossover command (in Data Analysis) the situation has evolved to
handle three GROUPS of dimensions with the power to regroup them at will. I believe Crossover is Helm's original contribution and goes far beyond the power of various Pivot and CrossTab facilities that you find in other products. Not a "subterfuge" then, but a powerful technique. A WinHelm with N physical dims is still a nice future development, but far from urgent (we find). Having several "keys" on the labels is also a simple way to handle sparsity (many-to-many key pairs) without losing the advantage of direct disk access (no index building, Btrees, etc.)
Efficiency. We find that Helm can handle one order of magnitude more data than similar products. On a Pentium machine we have developed (with careful design) data cubes with 100 million data cells. All comparable benchmarks with OLAP or non-OLAP products are of interest to us---and we expect to have more than an honorable mention on this point.
We have the impression that you did not explore the high level cube primitives that HELM offers and that (thanks to APL) have no counterpart in other OLAPs. Examples are Crossover, Blowup, Disaggregate, Merge (in Data Analysis) and File/Create from flat file, load database contents/From HELM database (in Administrator).
Access to keyword languages and pure APL is a major feature, in our view. In the DOS version this is ubiquitous, in the Windows version it is confined to few places (e.g. Select/Command or Execute) but is still a vital feature to customize Helm with a few keyword answers (instead of writing code).
Co-operation with APL developers is an issue that interests me personally. An Open Source policy in my view does solve much, because we want APL people to build on top of existing services, not fiddle with them to customize them. Example: the knowledge of 4 global names (T, DI, DA, DB) is enough to add a new cube operation that correctly interacts with all existing code. But many more interface points should be available, existing APL functions (of all granularity) should be documented for safe use by third parties, etc. I am looking for good ideas in this area. As you note, the LINUX co-operation is unique in history, and very difficult to clone...
Return to top
Copyright © Dick Bowman 1999-2013; latest update 8 March 2013 (repaired links)
Back to the J\APL Contents Page