How The Public 8Ball Works

A video camera looks down on the ball from above and sends its signal to an STB TV/PCI card in this computer where this program digitizes the frames, converts them to JPEG, and sends them out to you as a multipart/x-mixed-replace type of content. This is a slick technique since it lets the server program pipeline nicely staying just one frame ahead of your browser. It automatically runs at 1 frame per second or so on a 28.8 modem and up to 10 frames per second with faster links.

For those of you hung up on the picture, the objects are, in order...
a set of UHF/VHF rabbit ears, the head of the lamp, a rubber band, the camera and its cord, the 8ball in its lego cradle, the IR tranceiver, the Lego RCX controller, my keyboard cable, the top of a tiny Boston Acoustics speaker, the power supply of an Apple Studio Display, Patty Larkin's Angels Running CD, and the base of the lamp.
Everything in the picture is outstanding, except for the rabbit ears which stink, the camera which was real cheap at our local surplus electronics store, and the lamp which is unremarkable.

And yes, that is representative of my entire office.

How It Doesn't Work

IE hasn't gotten around to multipart/x-mixed-replace support yet, so you should use Netscape to view the 8ball if you swing both ways. I waited a year and a half for Microsoft's legions of programmers to get around to implementing x-mixed-replace, but I finally gave up and implemented a Javascript abomination to do the job.

They make it hard. The 8ball needs to communicate state changes to direct the rate of image loading. Various IE versions violate the Javascript spec in different amusing ways. I finally found one mechanism that works on all known current versions that lets me send 1 bit of state change. It involves delete images out from under the browser and catching the debris with an onerror handler.

Other things I've tried in the long unhappy path to IE Windows support...

• MPEG-1 encoding. I can encode MPEG at about 6 frames per second on this CPU into a stream that will fit down a modem. Sadly MPEG-1 is restricted to 30 frames per second so I can't stream it. (I'm not sure windows takes a stream either).
• MPEG-2 encoding. I have an MPEG-2 encoder, but it is too slow to use.
• Quicktime. I found a quicktime library for unix that lets me encode as a series of JPEG frames. Unfortunaely the library is flawed and generates quicktimes which are illegal on Mac and just hang windows. update: later versions of the library no longer make illegal movies, but it can't generate streaming movies in realtime.
• Flash. I'm now looking at Macromedia's Flash format. Its intended as a vector animation format, but it appears to support JPEG bitmaps.
• Also under consideration is a contorted client-side pull application, but I'm not sure IE will be able to stop pulling when the image ends. update: This is the one I finally made work.

Just Plain Neat Things

I send the frames at low quality except for the final frame, which is sent at a medium quality. You get 30% faster frame rates during motion but still get a reasonably nice picture in the end. update: Its even neater now. I send reduced resolution images during the ball spinning portion then progressively increase the quality as the image stabilizes.

It only takes about 10% of my PII-266 to run this thing.

Everyone gets their own destiny. If someone is using the 8ball you will be queued and get an image showing your position in the queue. This is actually some of the trickiest code involving semaphores, shared memory segments, signals, and forking. Yes, there is still a tiny race condition related to number assignment, but I can live with that. Its just possible that two people might get the same number. They still get handled in order and get a unique destiny.

How has it Failed?

• There were a number of early hardware failures when the 8 ball would slowly rotate in the cradle until the light sensor could not see the white dot with the 8, this resulted in perpetual spinning. These were finally resolved by sticking tiny centering pads directly to the 8 ball inside the axle mounts.
• After approximately 90,000 questions (over a half million ball reversals) the 8-ball hardware failed. One of the two redundant motors ceased to function. The ball continued in operation with a single motor and somewhat erratic centering at the end of the cycle. I do not expect to replace this motor, but rather to drive two 8 balls off the same RCX controller, each with a single motor.
• After approximately 100,000 questions the remaining motor failed (probable bearing failure) putting us out of commision. I reworked the shaking code to be lower impact on the motor, it now coasts to a stop then reverses direction rather than reversing under motor power. We have continued operation with a borrowed motor. Lego has been asked to supply replacements with no response. The guys at Arrick Robotics have offered some more robust positioners. Look for an industrial quality, multiball shaker in the near future.
• On July 20, 1999 slashdot mentioned the 8 ball and the resulting traffic overwhelmed the server. The 8 ball can only serve about 400 visitors per hour, (about an 8 second cycle time) we were receiving 1200 visitors per hour. The waiting queue went over 70, the resulting system load caused the server to assume something was wrong and reboot. Timely intervention resulted in a modification to the nabber program to shun visitors when more than 20 people are already in the queue. This takes the form of a graphic informing them to try back later instead of the usual "You are 2345, now serving 2356" messages. Since this modification there has been no problem with system load. Even with the limit of 20 the ball remained saturated until hours after it had moved off the slashdot page, 2/3rds of the visitors waited more than 100 seconds for their turn at destiny.
• There were some instances where the nabber program would stay with a visitor for more than 10 seconds, even up to many minutes. This was traced to a 'flush' in the exit handlers of the C runtime library. Even though the watchdog timer had fired and terminated the nabber it could be held up if the visitor's IP address ceased to respond to any TCP traffic. This was resolved by forcing the output file descriptors closed before exitting so the flush calls would fail immediately.
• A number of users accused the site of being a fraud because they always got the same destiny. Despite my best efforts to put no-cache tags all over the HTTP and HTML layers some ISPs seem to have aggressive transparent proxies. I believe I have circumvented this by putting a random parameter on the image movie URL so it won't match their previously cached URL.
• The 8 ball slow developed a problem where it would center incorrectly. This may be related to motor wear and rotational speed, but was certainly caused by seeing a bit of the 8 and mistaking it for the edge of the white dot. I solved fixed this by moving the optical sensor further from the ball so the black lines of the 8 can no longer have enough effect to be detected as false darks. This also involved changing the light/dark thresholds in the code.
• John Brockus pointed out that I had altered the 8 ball code but not update the code hanging off this page, so I have now altered the build process to automatically regenerate the HTML code you see everytime I deploy new code.
• I run the unit off of a wall wart power adapter. When there is a power failure the RCX forgets its program and must be reloaded. I'll move it to a UPS.
• 184,000 fortunes and I've burnt up another motor. I've switched to Radio Shack motors. $4 each instead of $30 for Lego, plus its a much beefier motor it should last longer. Unfortunately is also a high speed low power motor so I had to go through 3 stages of 5:1 reduction in Lego gears and the losses are tremendous. Its sounds like gravel in a blender.
• Reworked the system to use a worm gear for the first stage. Much quieter now, but the motor just barely starts. Its fine after it starts. We'll see how long this lasts.
• In response to months of prodding by Tim Jensen I've added a final tiny jiggle to the 8ball motion and caused the 8ball to rest upside down when idle. The jiggle mostly eliminates 'edges', and the upside down rest position eliminates bubbles by getting them into the bubble trap.
• In the 'about d*mn time' department I have added code to switch the video to the proper input so you won't get screens of static every time I forget to flip the tuner back to the camera.
• We are up to almost 250,000 fortunes and the radio shack motors seem fine.
• We are now up to 400,000 fortunes and doing well. I'm still struggling to support Microsoft's Internet Explorer. Its just unbelievably difficult to do live streaming video on that platform without using a Microsoft server. I came close today. I can make it work in a primitive way with Javascript, but I can't handle queueing during periods of heavy use. Back to the drawing board.
• October 7, 2000: 460,000 fortunes and I finally can support internet explorer! The server side code is fiendishly complex, the client side javascript is fiddly, the frame rate rather stinks compared to the nice streaming MIME, but it works. Now maybe I'll stop getting hate mail from Microsoft's flock.
• Jan-Feb, 2001: The 8 ball has suffered multiple systems failure. The Gateway computer that ran the 8 ball became flakier and flakier. It finally succumbed and will no longer get of if BIOS. The robot cradle has worn through the lego bricks again causing slippage. I fixed the cradle with a washer and moved all the gear to a new computer. (A PIA from thelinuxstore, $350. I love cheap hardware.) The frame grabber card won't work reliably in the new computer. (I hate cheap hardware.) It drops about half the pixels in the PCI transfer. I've switched to a cheap Kensington USB webcam. Its working nicely except that it does not deal well with the halogen light I use on the 8 ball. Very bad color rendition. I suspect it is seeing too much infrared. I tested a white LED from my keychain and it is beautiful color. Hopefully RadioShack will have one and I can wire up a light this afternoon.
• Aug, 2001: The 8ball continues its uncanny streak of breaking everytime I take time off out of town. This time the PIA server had RAM failure. Lesson in RAM reliability: There is good ram and there is bad ram. The guys selling it know the difference. (They won't tell you.) This ram came from a lot of 6. One was bad immediately. Over the course of a year or so all of the remaining 5 have now failed. Spend the extra 20%, get the good stuff
• Jan, 2002: 1,800,000 fortunes and we've worn out the Radio Shack motor. Feels like the bearings are siezing. That lasted about 10 million ball reversals for $4. Not a bad deal. I've got to run down and get a new one. I'd rather not. I've been avoiding the Shack after the last couple pieces of gear I bought there were complete and total crap. But, if I get the same motor from them I won't have to redesign the gearing and cradle again.
• Jan, 2002: The new motor from Radio Shack is crap. It was assembled poorly and will not run. I have rebuilt the old motor by cleaning its contacts, replacing the brushes with the ones from the new motor, and relubricating the bearings. It turns out the bearing alignment is critical to devent performance in these little critters. The first two times I reassembled it, the motor ran with significantly reduced power until I worked out a bearing alignment procedure. The rebuilt motor is working, but is a bit noiser than I like. I'll chalk that up to new brushes and hope it quiets down soon.
• Feb, 2002: Another new motor. The rebuilt didn't last long. I then tried a motor with gear box from a cheap electric car, but the gears wore too badly. This was followed by a shamefully long outage while I thought about how to redesign the 8ball robot. I was trying to use this fantastic pair of motors from a Digital TK50 tape drive, but they were just too good. If the eye quit working and the ball ran away there was too much power and things would be broken. If I built the cradle sturdily enough to withstand the motors it lost its charm. Moping commenced.
• Sep 4, 2002: Found a great little 24v dc motor with a nicely machined gearbox at Gateway Electronics. It runs with just a whisper of noise and has the best jerking torque of any motor yet for minimizing edges. I still need to attach the lego shaft better (loaned out my soldering iron to a software engineer) and align the shaft a bit better, but I think this motor will be good for a couple million fortunes.
• Oct 5, 2002: Shaft failure. I had taped the lego shaft to the motor shaft and the adhesive has turned to goo letting the shaft spin. I have retrieved my soldering iron. Doc Sewell did no lasting damage to it, himself, or the universe in general, quite a surprise for loaing a soldering iron to a programmer. I have used some layers of shrink wrap tubing to join the lego shaft and the motor. I didn't have one quite the right size for the first layer on the motor shaft. We will see.
• Oct 14, 2002: Shaft failure. There is slippage in the motor shaft to heat shrink junction. May the powers of engineering forgive me. I have mummified the entire joint in very tight twist ties to increase the pressure. It works, but I am ashamed.
• Oct 15, 2002: The electric eye keeps seeing the black bar in the middle of the 8 and mistaking it for the edge of the dot leading to a slight misleveled position. This causes a dramatic increase in "edge" results. I have added a little indicator arm on the end of the cradle and moved the light sensor out there. The only potential problem is that it may creep out of sync with the ball.
• Oct 16, 2002: Ok, that didn't work. It crept 180 degrees out of phase in 24 hours. New plan. I have stuck a white mark on the side of the 8 ball and moved the sensor to watch that. I built a little shield around the sensor to block ambient light to keep my light and dark levels more predictable. We are centering very accurately now. So much so that I lowered the camera two lego bricks to get a better view of the writing.
• Mar 10, 2002: Finally replaced the motor. That nice 24v motor with the great gearbox failed back in December. I tried to rebuild, but the coils are shorted and I'm not up to rewinding it. I pondered and pondered, but haven't come up with a better solution, so I am resigned to feeding the 8ball a stream of motors. I currently have two stages of 5:1 reduction balanced lego gear reduction and a rather standard, ungeared 12v motor on them. It growls more than I'd like, but it is working. The motor is $2.95 to replace from Gateway Electronics. I can swap that a couple of times a year as needed.

You can watch of movie of the ball dispencing mysticism if you like. (2MBytes MPEG-4 file. It is typed as quicktime since windows falls on its nose if you tell it MPEG-4).