BitScope Control Plane

It is possible to open the console as mute. This means the console will wait to receive a certain number of characters before it sends any characters back to the master. This is useful for error recovery when the remote host has "gone rogue".

The command to do this is the same as opening the console but with an additional specifier that tells the console how many characters to receive before enabling transmit. The high digit of the baud rate specifier is used for this as follows:

Up to 15 "mute echo characters" can be sent this way. This is usually more than enough to recover most bootloaders and operating systems.

Each node has a unique (geographical) location (within a cluster) at which it may be accessed. This location is referred to as the node's station (or ID in shorthand). Its default value is assigned in hardware. Literally it is address of the physical location of the node in the cluster. Its value may be modified by a configuration setting under certain circumstances.

Two registers report the ID for each node's BMC:

The vmStation is assigned the value of vmIdent at boot time unless (re)configured:

In this case the vmStation will be assigned the value of the configuration variable 7e, unless the blade is mounted in a BitScope Cluster, in which case the vmIndent value applies which cannot be overriden.

The BMC detects whether the Blade is physically located is part of a BitScope Cluster by checking the node ID. When a Blade is not part of the cluster, the ID for each node is virtual and assigned the following defaults:

Where N is an arbitrary node number within a cluster and ID is the station, i.e. the node's BMC address on the control plane. When a Blade is part of a cluster the ID will be automatically set to be different to these defaults. The ID for each node depends on the physical cluster and is immutable based on how the hardware has been commissioned.

When the BMC detects a node with one of the default ID listed above, it knows the blade is operating stand-alone. In this case, the default ID may be used as is and this is recommended.

When a blade is part of an ad-hoc cluster (i.e. custom cluster built without BitScope's cluster infrastructure) which comprises more than one blade, the ID of the nodes in the blades must be configured to be different to each other to avoid goegraphic address (i.e. ID) collisions on the bus. If you don't do this, all blades will locate their nodes at the same four addresses on the bus; 7c, 7d, 7e and 7f. This of course results in collision. This is an ad-hoc cluster commissioning responsibility which should be undertaken, one at a time, on each Blade before the blades are interconnected into a single cluster. This need only be done once and only when building ad-hoc clusters.

There are a range of potential error conditions. The set of errors that are detected are:

Any error is reported via the vmError register.

The vmError value must be manually reset to 00 if you seek to detect whether any error has occured. However, none of these errors are serious so all can be ignored (simply retry the operation if it did not succeed the first time). Note: errors prior to the one reported may have occured. However, such errors are undetectable (only the most recent error is reported).

The BMC can be reset. This is not recommended in normal usage but it can be useful to reload configuration parameters (perhaps after changing them) or to recover normal operation when uncertain as to the state of the BMC. The ! command activates reset. However, as a safety precaution, it will only fire if the value 55 is loaded to the vmInput register (i.e. issue [55]! as a command sequence).

The Meter M command is used to measure voltages, currents and temperature. An opcode is specified upon issing M to select which signal measure. The CODE returned as an unsigned Q16 with the following meanings:

The RANGE column reports the full scale value of the measurement. The VALUE column reports some typical example measurements.

The Mailbox is a single byte exchange.

With appropriate software running on the master and slave it can be used to create graceful runtime management protocols for use with the cluster control plane.

The Mailbox provides atomic byte exchange between a slave and master across the control bus. When a master has opened a pipe and connected with a slave, this command allows it to send a byte to the slave and receive the most recent byte issued by the slave (using the same command).

On either the slave or master, typical usage is issue XX` to send XX into the exchange. The exchange atomically replies YY as the reply data to the ` command. YY is the most recent value issued by other side of the exchange.

The mailbox command ` is idempotent so it may be issued multiple times. Subsequence issuance of the command is used simply to return the state. It would normally be used with a polling mechanism on each side of the mailbox. For example, to implement a management protocol analogous to UNIX Init, a set of master "mailbox tokens" could be defined:

These would be issued by the cluster manager (master) and received by the node (slave). A set of matching slave state tokens could be defined as:

The master can issue STATUS at any time to ask the slave for its state. When STOPPED the master knows it is safe to POWEROFF of the node. If the state never reaches STOPPED (either because the host has crashed or the host is not running software to report this to the BMC) after a (master defined) timeout, the master can POWEROFF the node.

The SRAM is region of memory shared between a node and the cluster manager.

A master can use it to communicate arbitrary data with a slave. In this case the mailbox can serve as a semaphore mechanism. Alternatively, a slave may use SRAM to record information which the master may interrogate later. For example, a console debugging or logging port on the slave.

How SRAM is used is not defined in this document. How it is read and written is defined here.

The R, W and S commands are used to read, write and dump SRAM:

The + version of the R and W commands can be used in sequence read or write successive values in SRAM. The - may be used to decrement through SRAM if preferred.

The EEPROM is used to store persistent state.

How it is used is defined in Configuration. How it is read and written is defined here. The r and w commands are used to read and write EEPROM:

The + version of these commands can be used in sequence read or write successive values in EEPROM. The - may be used to decrement through EEPROM if preferred.

The Coefficients C command is used to read the BMC calibration coefficients. An address precedes the issuing of the C command to select which coefficient to read:

They are useful to calibrate M results for voltage, current and temperature meaurements to better than default precision. This is not required for normal operation. A separate application note will be linked here with details about how to do this when available.

I've followed the connect and unlock procedures but the master node BMC does not respond.

Why not?

How do I check if it's working?

The recommended way to check the status if unknown is to issue the = command.

If there is no response the BMC is probably locked.

However, there is another possibility.

The BMC may be unlocked but the master node may have the console open which is connected to a slave node. The master is therefore talking to the slave node. If that slave is not responding for whatever reason, e.g. no system console enabled, operating system has crashed, then you will receive no response to any commands you send even though the master BMC is unlocked.

The same thing can happen if you select a non-existent slave node, usually by specifying an incorrect node address with the { or | commands. To escape from these situations send } and/or ^G to escape back to local mode and then issue = to confirm you have succeeded.

If there is no response to any of this, then there may be a serial I/O problem talking to the local BMC or there is a hardware fault on the control bus or BMC.

Have you re-checked the tips explained in the connect section?

Second production release.

1. fixed 115k baud console.

   Previously the remote console feature occasionally dropped characters when a lot of data was sent from the remote host via the bus. This could occur even if the remote host was sending at precisely 115k baud. This problem is fixed and Cluster Blade can now accomodate abnormally high rates. The local host (master) may not support this however so the caveats still apply.

2. unlock code is changed

   The previous (default) unlock sequence was BitScope. This has been changed to UnLockMe. This is just the factory default, it can be changed to whatever is required.

3. relocated some API registers

   These changes made to simplfy the API and to accomodate changes made in support of (1). The body of this document has been updated accordingly.

Interim update (not released publicly).

1. fixed fan idle algorithm.

   Previously, when a fan was idle, a low level "jitter noise" may have been emitted. This was audible for some people (those with particularly good hearing). Whether the noise was emitted depended on cpu load, the type of fan and the angular position of the fan when idle. This has been fixed. When idle the fan is completely silent.

2. enabled fan when console open.

   Previously the built in fan control was not active when the console was opened. In this case the fan speed would not change until the console was close. Benign in most cases, it did mean that if one ran a heavy workload via the console, the fan would not increase speed until the console was again closed. While closing the console after issuing a command is normal practice in cluster management, this change means the fan speed now continues to be managed even when the console is open.

3. enabled fan when master active.

   Similarly to (2) the fan speed is now controlled when a node is being used as the cluster manager. While it's not common to run a heavy workload on a node that is the cluster manager, it is now possible to do so and have the fan respond correctly.

Interim update (not released publicly).

1. fixed fan control algorithm.

   The fan speed is driven by the node current draw, subject to scaling, and saturates at a point set by the fan control parameters. This worked for 02 (x1) and 03 (/2) but not for 00 (x4) and 01 (x2). In the latter cases, the fan failed to increase speed when current draw went beyond 1A (with 00) or 2A (with 01). While this could be worked around (with manual override) the fact that a node can draw up to 4A meant cooling may be insufficient in high load use-cases. Fixed.

Interim update (not released publicly).

1. added open console with mute function

   The baud rate specified is now limited to the low nybble. The high nybble specifies how many characters the console must accept from the bus before sending data to the bus.

   This mechanism allows a bus master to "force feed" up to 15 bytes to a slave host (via its console) before the slave host is allowed to send data back to the bus (via its console).

   The canonic use-case is a host vomiting data (via the console) to the bus for which a ^C or ^D is required to stop it, but, because the (simplex) bus is full of console traffic, the master can no longer reliably transmit a character back (over the bus) to the slave (because of bus collisions). Collisions will cause all nodes to drop off the bus but reconnecting to the rogue slave results in the same problem (the slave will vommit onto the bus as soon as the console is reopened).

   By opening the console with mute, the master ensures it can send up to 15 characters to the slave via the bus before the slave has a chance to flood the bus with (more) junk. Upon receipt of the terminator (^C or ^D) the slave host should stop vomiting allowing normal operation to proceed. If it does not, the slave has gone completely AWOL and nothing the master can do can retrieve it (except avoiding opening the console). In this case, a hard reboot (power cycle) may be the only option to recover (as the problem is within the slave host).

2. removed extraneous LF from / and \.

   This was a bug in CB04A016. Now fixed.

3. modified pipe to use | and { uniquely

   • When opened with | it is closed with the Close Pipe.

     This is intended for interactive use. The default Close Pipe is ^G. It may be configured to a different character (via Close Pipe). This may be necessary if ^G is used by a slave node (via the console) for some other purpose.

   • When opened with { it is always closed with }.

     This is intended for non-interactive use. Command strings are more readable using this mechanism. The closing character cannot be replaced (it is always }). This means it may not be appropriate for use with a console (if the host uses } for other purposes).

   In both cases, a closing character is returned when the pipe is close. Closing the pipe causes all attached nodes to detach. On the master the ^G or Close Pipe or } character will be returned (depending on how the pipe was opened and how it's configured). On the slave the ^G or Close Pipe character will be returned. In this case } will never be returned because the slave never opened a pipe.

   Normally there should only be two attached nodes (the master and slave). If there are others (which an error condition) they will detach when any master closes its pipe or any node attempts to become master. This means the bus is always recoverable. These (extraneous) nodes will return ^G or Close Pipe to their hosts unless they (also) had a pipe opened on the bus with { in which case } will be returned.

First Production Release.

1. added the host console mechanism. The Console allows the master to talk with a slave host via the bus. The console command is ~. The value in the input register selects the baud rate as 00 115k 01 9k6 02 19k2 03 57k6. The 115k baud rate does not work reliably for strings longer than a dozen characters (due to bus overrun). This may be fixed in a future revision (with fifos). Baud rates 01, 02 and 03 are recommended until then.
2. defined unlock default string BitScope
3. added echo squash for console use. When an open pipe sees the console command '~' it enables squashing. When squashing, characters sent by the pipe are not echoed. The exception is end pipe (an address) which must echo (to end the pipe). This mechanism is added to enable the use of a console.
4. modified end pipe mechanism. The Close Pipe character (^G by default) is no longer sent (or received) over the bus. When received (from the host, on the master) the master puts its own address on the bus instead. All nodes (including the master) detach from the bus upon receipt of an(y) address. The (releasing) master node does not (re)attach (itself) despite its own address appearing on the bus. Any consoles (if running on any slaves) also terminate and detach from the bus. No character received by a console (from its host) can cause the console to terminate. All 7 bit characters (except the Close Pipe) are sent to a slave's host (via the console).
5. modified so high bit set characters are dropped Only 7 bit ASCII is legal across the bus. High bit set values are reserved for address selection on the bus.
6. removed Close Pipe command (i.e. * is no longer used or required)
7. updated pipe open can use | as well as { They are equivalent to each other to open the pipe. They close differently.
8. removed / \ reply character payload Use state command = to learn state

Second beta release.

1. added serial unlock mechanism.

   The default unlock word is BitScope.

First beta release.

1. added peon mode.

   A node may be Peon which means it does not have permission to become a cluster manager (Master) or to change its own configuration. This will be the case in a managed cluster by default because configuration is the exclusive domain of the cluster manager. Asserting Peon Mode in an unmanaged cluster achieves the same result. A node can always read its own configuration so it can know how it has been set up to run. To disable Peon, No SRAM and/or No Fan the master must change parameter value to something other than aa.

2. fixed serial error reporting.

   It is sometimes possible for a host to overrun or experience other low level serial communications errors when talking to the BMC. This can occur if the VM protocol is violated by a user operating interactively or by bugs in programs talking to the BMC. Such errors are non-fatal but they need to be reported (for diagnostic purposes). This update fixes the reporting of these errors as documented here.

3. relocated measurement, cooling and calibration commands.

   These commands were previously implemented as vectors. They now exist as commands which makes them easier to use interactively.

4. fixed bus recovery mechanism.

   It was previously possible to lose control of the bus when illegal characters were sent to the BMC. This is no longer possible.

5. added address override mechanism.

   It is now possible to relocate the address of nodes other than by the hardware mechanism built into BitScope Clusters. The new address is store in FLASH. This allows the creation of ad-hoc clusters comprising more than one Blade without requring the use of hardware addressing.

6. protected the reset command.

   The reset command ! now requires a uniqu vector 55 to be specified before it does anything. This change make inadvertent use of reset unlikley. Reset remains a benign operation (from the host's point of view) unless boot with power off is enabled (in which case the node will power off - caveat emptor).

7. added boot with power off feature.

   It is now possible to configure a node to remain powered off when blade power is applied. The makes it possible for a cluster to be configured to automaticlly power on only those nodes that are enabled to do so when power is applied to the cluster. That is, any "boot with powered off" node must be powered on by the cluster manager before use.