The Resurrection of bWatch
When the world was dominated by dinosaurs, a new beast arose from the depths: one that relied on the earth's community to grow and thrive. It was Beowulf. No, not the Scandinavian warrior, but an approach to high-performance computing (HPC) that uses common x86 processors, conventional Ethernet networking, the Message Passing Interface (MPI) or a parallel virtual machine (PVM), and Linux.
With Beowulf, everything was new. Previous HPC systems all had proprietary tools to manage and monitor servers (nodes) in the cluster, so the tools for Beowulf clusters had to be developed, including those to monitor clusters. During the first few years of this Clusterian period of HPC, one of the key requirements was a simple tool to monitor nodes – something simple that would give the status of all nodes in the cluster.
A visual presentation of the load on the nodes (with the uptime command), node uptime, and maybe even some memory usage would let you determine the status of a cluster with a quick glance at the screen. At that time, lots of people were experimenting with and developing clusters, so many were homemade and small: A 60-node cluster was considered large, so you would easily be able to see all of the nodes on one screen with a little scrolling.
bWatch
Python was still in early days, but Tcl/Tk was very popular. The high-level, general-purpose, interpreted and dynamic Tool Command Language (Tcl) was created by John Ousterhout in the 1990s. From the name you can probably tell that it was intended to be a language for writing tools for whatever platform you were using. (I will begrudgingly admit there are more platforms than Linux and other Unix-like operating systems.) Because it was interpreted, you didn’t have to change code and recompile constantly. You could just change the code and run.
Ousterhout also needed a tool for developing graphical user interfaces (GUIs) for Tcl applications, so about one year after starting Tcl, he created Tk, which has expanded beyond just Tcl with bindings to many other languages. Tk provides widgets commonly used for desktop applications such as buttons, menus, a canvas, text, labels, and so on.
To create something useful for Beowulf clusters, Jacek Radajewski used these two tools to write a simple monitoring tool, bWatch . Although simple in concept, when first released, a huge portion of people using and developing Beowulf clusters used it. The first tine I used it, I was very excited because I could get a visual representation of the state of the cluster (Figure 1), rather than having to log in to each node or use a command-line tool to run parallel commands on all the nodes and parse through the data.
bWatch concepts are pretty simple: The idea is to gather information about the node by opening a shell on that node and running Linux commands. Originally, bWatch used the remote shell (rsh ) to execute shell commands on another node. Although rsh is very insecure (it transmits everything without encryption), at the time, it was commonly used, so bWatch used it.
bWatch shells to the node and runs the uptime command to find the current time on the node, the number of users, how long the node has been up, and the 1-, 5-, and 15-minute load averages on the node. Isn’t it amazing how much information is contained in this one command?
bWatch also runs the command
cat /proc/meminfo
to gather information about the memory being used. With the output from just this command and uptime , you can get an idea of the status of the node.
Running bWatch
One recent morning, I saw the word "bWatch" on a dry erase board in my home office. Evidently at some point in the last five years or so, I wrote that, but I couldn't remember exactly why. Regardless, that morning I decided to try to get bWatch running again on today's Linux and hardware – or at least give it a good try, even though my Tcl knowledge is gone.
The last version of bWatch posted is 1.1.0a with a date stamp of June 15, 2004, which makes it about 21 years old now. There's no chance this will work, right? I wanted to try it anyway, but I will probably have to make some changes along the way.
wishx and ssh
bWatch is written in Tcl, which I didn’t remember at all, but it’s fairly easy to read. The first line in the bWatch.tcl code is the shebang (#!/bin/ < something > ) that points to the command or tool the code is supposed to run. In the case of bWatch, it is #!/user/bin/wishx (Windowing Shell), a TCL and Tk interpreter. However, wishx has since given way to just plain wish , which I easily installed on my Ubuntu 24.04 system; then, I changed the interpreter from wishx to wish on the first line of the code. Easy enough.
bWatch uses rsh , so I knew I had to change that, as well. I searched through the code and rsh is defined in the line,
set command(rsh) "/usr/bin/rsh"
so I changed the set command target to ssh :
set command(rsh) "/usr/bin/ssh"
Everywhere else in the code, rsh itself is not used, only set(command) .
.bWatchrc.tcl File
The next issue was how to specify the node names to be used, which meant I had to check the README file, where I found that the list of nodes (hosts) is specified in a file named .bWatchrc.tcl in the root directory:
set listOfHosts {node1 node2 node3 node4 node5 node6 node7 node8}
I’m not sure what restrictions are imposed on node names, so I just used the output from the command hostname on all nodes. (I only used one node for this article.)
First Run
To install bWatch, use:
sudo make install
This command puts the code in /usr/local/bin , which is also in my default $PATH ; however, you might check that it is there. Next, I ran the command bWatch.tcl (Figure 2). Everything sort of looks good, but notice that the Shared Mem column has no values. Something is amiss.
When I ran bWatch, a wish console window popped up (Figure 3). The complete error message reads,
getSharedMemory(laytonjb-Precision-7680):child process exited abnormally
so something is going on with the getSharedMemory procedure.
Before digging into this error, I tried turning off data collection for getSharedMemory by clicking on the Options button, deselecting the Shared Memory option (Figure 4), then clicking Apply and OK to close the window.
At the top of the main bWatch window, selecting Refresh causes bWatch to update the information for all nodes possible (Figure 5).
Back to Shared Memory
Although missing just the shared memory information is probably not a big deal, it would be nice to get the data originally captured in bWatch. The procedure that gets the shared memory information in bWatch.tcl is:
proc getSharedMemory {host} { global command bWatchDir set errorCode \ [catch {set sharedMemory \ [exec cat $bWatchDir/bWatchMemInfo | grep MemShared]} errorMessage] # check if the command returned an error if {$errorCode} { set date [getDate] .console.messageList insert end \ "$date :: getSharedMemory($host) : $errorMessage" return "error" } else { return $sharedMemory } |
Notice that it runs the command grep MemShared from the information gathered with cat /proc/meminfo . (You need to dig into the code a bit to find out that it stores the result from cat so it can be searched for the needed information.)
The output of cat /proc/meminfo from a modern Linux system won’t show MemShared . The closest output is Shmem , so you can just change the grep command to grep Shmem . If you do this and restart bWatch.tcl , you’ll see that the Shared Mem column reappears (Figure 6). Note that you might have to go back and re-select Shared Memory under Options to get it to show.
Summary
I remember using bWatch many years ago on my clusters, and I loved the simplicity of the GUI and its ease of configuration, along with the simple information it provided. After 21 years, I was surprised how easily I got bWatch working again. I think that shows the importance of keeping programming languages and GUIs somewhat fixed, along with LInux interfaces (e.g., /proc/meminfo ), with a few small exceptions. I also like that you don’t have to be an admin to use bWatch. Even a humble user can gather this data.
Give bWatch a whirl and poke around with some of the options (which you can save to your .bWatchrc.tcl file). Although there really is no need, I think it would be interesting to port bWatch to Python, because it would probably be faster than I relearning TCL/Tk.
Subscribe to our ADMIN Newsletters
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Most Popular
Support Our Work
ADMIN content is made possible with support from readers like you. Please consider contributing when you've found an article to be beneficial.
