As best as I can tell, I am in what I believe to be a unique situation:
I am the maintainer, sole developer, and probably the only active user of the programming language SPITBOL.
(If you know of anyone else who is maintaining an open-source implemenation of a programming language that has only one user, please let me know, via comments to this post.)
Let me explain.
In June of this year I took over maintainership of Macro SPITBOL. Mark Emmer had labored for almost a quarter century in this task. He kept Macro SPITBOL alive, and that is the task I face going forward.
Macro SPITBOL, the work of Robert B. K. Dewar and the late A. P. “Tony” McCann, is the best implementation of the SNOBOL4 language yet created. I believe it to be a remarkable accomplishment.
SNOBOL was created at Bell Labs in the early 60′s. The basic reference is The SNOBOL4 Programming Language, by R. E. “Ralph” Griswold, J. F. Pogue, and and I. P. Polonsky. It is known to SNOBOL4/SPITBOL folks as just “The Green Book,” due to the color of its cover. (Mark Emmer secured the rights to make it available, and the pdf for the book is freely above at the cited URL.)
SNOBOL was the first language to address processing strings and text in a serious way. It’s most notable innovation was the notion of “pattern.” As part of this, it introduced many primitives such as BREAK and SPAN. Those terms are now commonly used in many libraries and programs that process text.
SNOBOL was one of several languages widely taught in “Introduction To Programming” courses in the 70′s and early 80′s.Then — I think it fair to say — SNOBOL sank into obscurity.
Macro SPITBOL was created in the mid 70′s. Dewar and McCann finished their work by the late 70′s. I did the port to CDC 6600 in the late 70′s, and worked with Robert to do the port to the IBM/PC 8086 in 1983. Mark Emmer then took over the project, and did ports to Mac, Solaris, and various versions of Unix.
At most ten people worked on refining the SPITBOL implementation. I’m the only one still working on it. Mark lends a hand now and then, but more as an advisor than programmer.
Since taking over maintainership I have finished the Linux port (Mark had already done most of the heavy lifting). I started writing about SPITBOL on this b log to drum up interest, and I am actively working on the port to Apple’s OSX, to be followed by a port to the ARM architecture, so that SPITBOL can be made available on Apple’s ioS and Google’s Android operating environments (both use ARM chips).
Here is where the work stands:
- There have been just over 3,000 downloads of the Windows version since it was released in open-source form in July 2009. I’ve received less than ten, if not zero, emails and blog comments about this version since its release.
- There have been 30 (thirty) downloads of the Linux version since it was released two months ago. I’ve heard back from few than five people that they have downloaded it and also tried it.
- While a few (under 10) folks have expressed interest in the work, no one has stepped forward to work on the code as a developer.
- I’ve seen no sign anyone is actively using SNOBOL4/SPITBOL in a serious way. SPITBOL was once used by folks in the linguistics community, but I’ve seen no sign of recent work.
- I did a search on Google for “SPITBOL” in June and got about 17,000 matches. I also did a search on “SNOBOL4″ and got about 47,000 matches. I just repeated the searches and got the same results.
- Despite making many posts about SPITBOL on my blog, I have yet to see any one of them achieve more than 20 or so views.
In summary, though I know I have advanced the code, I have been a failure trying to drum up interest in the project.
What Next?
I am not giving up. I have several reasons to push on:
- SPITBOL is unique in the power it provides to manipulate strings and text. I’ve yet to see anything else come close.
- SPITBOL is both amazingly fast and compact. It can compile hundreds of thousands of lines of code per second, in a program consisting of under 20,0000 executable instructions. It runs at the hardware level. For example, it includes a compacting mark-sweek garbage collector, while I know of at least one widely-used programmng language that still uses reference counts. [1]
- The ARM implementation should be *very* fast.
- I have packaged the system so it is self-hosting. SPITBOL uses tcc for its C compiler, NASM for its assembler, musl libc for its runtime libraries, and Rob Lanbley’s toybox to provide basic Unix commands. SPITBOL is built with these open-source packages, and the source for them is included in the SPITBOL distribution, so you get not only Macro SPITBOL, but a complete development environment to boot.
- I believe there is a great opportunity due to the poor support for Unicode in other programming languages and libraries. If I can add full support for Unicode, I hope to realize “if you build it, they will come.”
- Perhaps most important: SPITBOL is fun!. I’ve been programming for over fifty years, and I have never had so much fun save when I was working on SPITBOL, as either user of the language or as an implementer.
- The quality of the system will speak for itself: Code Talks.
- Knowing that I am starting from zero, any progress will be great news, and it will be fun to see how far I can get. I had a similar experience working with Philippe Charles on Jikes. We started from nothing, and were able to build something that achieved some success. I hope to repeat that, and I know I will have fun trying to do so, even if I don’t succeed.
It is also evident that talking about SPITBOL is a thankless task for now, so I’m going to put off blogging, twittering and such, so I can concentrate on the coding, which after all is all that matters.
Most importantly, I believe that polishing the implementation and porting it to new environments is not enough.
The *only* way to move the project forward is to finish needed refinement and ports as soon as possible, and then to use the result to write applications in SPITBOL that demonstrate what can done when programming in SPITBOL.
I have a few ideas about useful applications, and hope to get to work on them as soon as possible.
We shall see…
Notes:
1. Can you name a widely-used contemporary programming language that still uses the 60′s software technology of reference counts to manage storage?
Tuxer's
Welcome
13 Comments
Hi Dave,
Just wanted to drop you a line and say (1) thank you for taking over this project and (2) I’m going to explore snobol.
I’ve been interested in it ever since watching an interview of Guy Steele mention that snobol was one of two or three languages that people should check out.
Clay,
Thanks for the note. What were the other two?
I wish I could remember what they were. I only remember snobol because I had never heard of it before then.
Python uses refcounting and GNU awk does too. I’m not sure it’s such a bad thing. It has advantages like freeing up stuff as soon as the last reference vanishes.
I used Spitbol as a high school student back in the day, so I remember it affectionately. I’ll probably play with the Linux version a bit. I found your blog in I guess 2009, when you were not actively posting or answering email, and I tried then to figure out what happened to you. I suspect you’re getting low views now because people stopped monitoring your blog after you quit posting.
I’m skeptical that an asm template system can produce code as tight as an optimizing compiler, and GNU C does have “goto *address” (used by GNU Forth), so a C implementation would be interesting.
Can you say how much ram it takes to run Spitbol? It might be interesting to run on midsized embedded computers, say with 64k, if it can run in that. There is a new ultra-cheap TI Launchpad board with an ARM M4 chip, with 256k of program flash and 32k of ram, so I wonder if that’s a possibility at all. I’m sort of interested in Spitbol now as a source of ideas about garbage collection in such a constrained environment. The language I’d be trying to use is more Lisp-like, though.
You say “20,0000 instructions” but I think you mean 20,000.
Paul,
Thanks for the note.
It’s not that reference counting is bad. It’s just that it has its problems, and is so inferior to the reclamation technique used by spitbol – mark-sweep garbage collection – which
finds more unused storage.
That was available in SPITBOL almost two decades before the first Python implementation appeared.
SPITBOL doesn’t need an optimizing compiler. It is hand-written to exploit the hardware to the max. Having worked on translators for CDC 6600, Intel 8086 and x86, I can attest it produces
optimial code for basic string operations.
As for size, SPITBOL/PC was 16-bit. There is also an additional implementation called MICRAL, by Dewar, Goss et. al. This work aggressively compressed the instruction stream. It was recently published, and can be found on Amazon, though the price is not cheap.
Yes, it should be 20,000, not 200,000. Current x86 version generates about 15,000 lines of executable assembler, plus 4,000 so so lines of data declarations and definitions.
This doesn’t count the C runtime, but by any contemporary standards SPITBOL is very small.
Yes, GNU/C does extend to allow ‘goto address’ but SPITBOL needs more than this. For example, it incorporates the notion of exception handling, something not found in C. I plan to write about this soon as time permits.
dave
I always wondered how the Incoterm implementation managed to compress Spitbol into its limited memory. The MICRAL paper seems to be available free (though with some typographic corruptions) at http://www.archive.org/stream/microspitbol00dewa/microspitbol00dewa_djvu.txt
As for optimal code, an asm macro system like MINIMAL combined with a peephole optimizer to eliminate redundant load/stores between templates seems to do quite well, since good register allocation has already been accomplished in the MINIMAL source. On machines with spare registers, like SPARC, hot variables are allocated to registers by the translator (something to keep in mind for a future x86-64 or ARM translator). Spitbol’s speed is proof that these techniques work, and any additional gains from better translation are likely to be (ahem…) minimal.
A C implementation would improve portability however, but there are issues that go beyond “goto *address”. Exception handling mentioned by Dave may be tractable in GCC because a function’s return address can be manipulated, but there are other problems. For example, garbage collection requires finding all pointers to live blocks in dynamic storage, and if C is hiding one in an invisible temp, storage regen will eventually fail. The MINIMAL source has other bizarre things, like an ordinal byte identifier embedded in the code section in front of each block of asm code, used (indirectly) to identify data types. There’s no corresponding construct in GCC, requiring that this mechanism be rethought. I made a brief investigation of the issues of recoding in C about 18 years ago, and didn’t see any mechanical way to do it, and so put it on the shelf.
Mark
Mark,
Thanks for the comment.
Re MICRAL, you can find a book about it on Amazon. It came out last year, and appears to be the papers produced by Dewar, Clint Goss (whom I knew whilst at NYU) and Marty Golumbic (who used to have the office across the hall from mine when he did a postdoc at Courant). I haven’t bought a copy since it is quite expensive. The MICRAL work is interesting as an exercise in aggressive code compression, though not of practical interest these days given the speed and available memory of even the smallest embedded systems.
As for an implementation in C, I agree that attempting to translate MINIMAL to C would be *very* difficult.
However, it would be interesting to have an implementation in C that interpreted MINIMAL code. Bellard’s success in writing a 386 emulator in Javascript that is able to boot Linux in a browser window in about ten seconds — a *very* impressive programming feat — suggests this code might be of value.
Indeed, based on Bellard’s work, it might be even more interesting to do an implementation of SPITBOL in Javascript.
thanks,dave
Having come accross your Spitbol-Linux, I am impressed: Thank you, very much. As a long-term user of the language (http://www2.hsu-hh.de/hisalt/projects/betautf8.htm), I would like to see runtime module creation, and above all, external function loading. Is there any way to give you any assistance or help?
Burkhard Meissner
Burkhard,
Checkout out your site — very impressive indeed.
And I thought I was the only person still writing SNOBOL4/SPITBOL programs … Good to see I’m not alone.
Re helping out, a long-term goal is to add *full* support for Unicode to SPITBOL. That will need 64-bit word, 32-bit character version plus suitable conversions for UTF-8, etc.
Re helping out, that will be the time for you to really dive in, though of course once the immediate porting efforts are in better shape, any help testing would of course be greatly appreciated.
thanks,dave
Dave – I’m deeply grateful to you for maintaining and developing Spitbol.
I’m a longtime Spitbol enthusiast (since the days of IBM360 Spitbol) – I use SPARC Spitbol every day, and it ‘just works’!
I suspect there are many more like me who quietly get on with using the language and only surface when there is a problem – and mostly there isn’t!
I wouldn’t have surfaced except that I have recently moved from Solaris 9 to Solaris 10 – and Spitbol no longer works. Has anyone else had a problem with upgrading to Sol 10?
best wishes, and keep up the good work – it’s much appreciated
I gave away my last Sparc machine a few years ago. If you want to try rebuilding the system, I can provide you with the source files. Contact me offline at marke at snobol4.com
Mark
Thank you for keeping Snobol4 alive in the wonderful form of Spitbol! I don’t do a lot of programming (I’m a writer/translator/editor), but when I do, it’s always to process text. My friends say I’ve gone retro, but I believe that in Spitbol, I’ve finally found the language I’ve been looking for. I downloaded your Linux version and have been getting acquainted with it for a few days now. I’m already quite certain that it’s going to prove to be just the right fit for my needs. Now I can stop jumping on this object-oriented bandwagon and that functional-programming bandwagon and get down to business–and enjoy it, too. Keep up the good work!
Gary,
Thanks for the kind words.
I will release a new version of SPITBOL for Unix shortly.
dave