Adding Graphical Users Interface to Graphing Temperature Measurements

I started a series of blog posts back in July about making temperature measurements with sensors using the one-wire interface to the Pi. I described the hardware, software requirements, and the python script I wrote to make and report the measurements.

Earlier this month, I added graphing capability using RRDtool and PyRRD. The python script is, of course, included. Prior to that post, I presented a tutorial on programming with RRDtool and PyRRD. The graphical results are pretty impressive. My only complaint concerns actually running the script. Before making any measurements, the user has to answer quite a few questions that appear on the terminal. Answering all of those questions, each time I run the script, got a bit tedious, so I thought about adding a graphical users interface, i.e. popping a window for those questions and answers. This post reports on those efforts.

Before proceeding any further, I hope you readers have taken a look at my post proceeding this one (Oct. 18). While developing and testing my GUI I ran into a serious problem. The script must be run from root because of access to the the GPIO hardware on the Pi. I found that Linux would not give me permission to access the graphical system running as root. Look at that post to see how that problem was handled.

When you launch the script here is the default window that pops up. I'll talk about why I mentioned default window in a little bit:

Default Window

Taking this from the top of the window, I'll discuss each of the widgets, in turn: I have two temperature sensors, one on the breadboard the other on the end of a cable. With the radio buttons I can choose either sensor, or choose both sensors. You might think two check boxes would be more appropriate than the three radio buttons. But, the radio buttons work better with the program that actually makes the measurements (the program that calls for the GUI window).

Next are two entry boxes for applying a legend to the graph for each sensor. The default has only the breadboard sensor enabled because of the radio buttons selection above, which is why the cable entry box is grayed out. Select cable sensor and the entry box for cable can be edited and the breadboard entry box is grayed out. Select both sensors and neither entry box is grayed out. If you don't type anything in the entry boxes the default text (as shown) will appear on the graph. RRDtool has its own rules and some may seem strange. Here, for example, if you wish to have a colon in your legend (cable: water temperature), you must escape the colon with the backslash character (cable\: water temperature).

After the legend comes the title for the graph. A default string is included. RRDtool does not require you to escape colons in the title, but does require you to escape spaces.

Comments are optional. You must escape colons here too, just like the legends.

Next we can choose the graph background color. Black, I think, looks good on the screen, but if you wish to send the graph to your printer, it would use an awful lot of ink. Consequently, we have an option of having a graph with a white (actually light gray) background, You can choose to create both graphs if you wish. In the future, I would like add the possibility of a custom color graph. That would have new windows popping up from the basic window, something I want to try coding. Like the situation with the sensor widgets, check boxes might seem to be more efficient, bur radio buttons work better with the script making the measurements.

The width of the graph is hard-coded to 600 pixels, but the height is programmable with the slider control. You can choose values from 100 to 400 pixels in increments of 100. A shorter graph is useful when the variation in the measurement values is small (nearly a straight, horizontal line). I have seen where RRDtool actually repeats values on the Y axis if the graph height is large and there is a small variation in measurement values. RRDtool selects the values on the Y axis, not the programmer.

Next, we have the number of measurements and the interval between measurements. The smallest interval being one minute.

Finally, we come to the matter of file names. As of now (meaning I may alter this in the future), the directory name is hard-coded in the script. I have a directory for all temperature measurements, but make up a new sub-directory, under that, based on the date (for example: 2013_10_25 for Oct. 25, 2013). The file name asked for by the widget is a base name without extension. Depending on the number of sensors used, and the number of graphs generated, we will make three to five files for each run of the script. The measurements used for the graph are stored in files with an .rrd extension - one file for each sensor. There is a .png file for each graph (one for black background, one for white). Finally, there is a .txt file generated to store all of the results, along with the measurement times, in an easily read form (the .rrd files are not easily read). This .txt file is another feature that is new with this version. If, for example, the base file name is rodger and we were to use both sensors, and generate both graphs, we would generate the following five files: rodger.txt, rodger_bread.rrd, rodger_cable.rrd, rodger_black.prn, and rodger_white.prn.

The check box to the left of the file name is to protect the files from being overwritten if they already exist. It looks in the directory with the current date. Putting a check mark in the box allows the files to be overwritten.

Let's put some values into the entry boxes but we'll make errors in all of the boxes:

Result Of Pressing Continue If There Are Entry Errors

After pressing the "Continue" button, the red error messages show up to the right of the entry boxes. The legend and comment boxes have non-escaped colons and the title has a non-escaped space. The measurement interval can not be zero (the number of measurements will give the same error if they total to zero). The error message will appear if a non-numeric character appears in the number of measurements or any of the measurement interval boxes.

There are three possible error messages that can appear to the right of "Base Filename". They are:

• Only numbers, letters, and underscore
• Will overwrite existing file ("Allow File Overwrite" not checked
• Must enter a file name (if you leave the box empty)

Once all of the corrections have been made, hitting the "Continue" button will kill the window and allow the program that makes and graph temperature measurements to proceed. If you hit the "Quit" button the window closes and the calling script terminates.

One other action happens before the window closes. All of the parameters entered in the window widgets are saved into a configuration file. The next time the program is run, all of the parameters are loaded into the window rather than the default values. This should save a lot of time if the same run is repeated or only a few changes are made. If a lot of changes are to be made a press of the "Default" button will bring up all of the default values.

Let's do an actual run. The cable sensor is placed into a 12oz. glass of hot tap water and allowed to cool to ambient temperature. Here is the window:

Window For Actual Run

After pressing the "Continue" button the window closes and you see the terminal window:

Terminal Window After a Few Measurements

The terminal window shows the parameters chosen on the GUI. It gives you a last look at your selections. If you decide you wanted something else, simply hit Ctrl C to terminate the script. A careful reader will notice the parameters on the terminal screen do not match the parameters chosen in the window above. That is because I forgot to save the screen shot, so two different runs are represented.

Below the parameters it says that the 1-wire modules had to be loaded. They are not loaded upon boot-up, so every time the Pi is powered up, or rebooted, the 1-wire modules must be loaded. The script checks to see if the modules are loaded, and, if not, loads them. Subsequent runs of the script will not need to load the modules if power stays applied and the Pi is not rebooted. Next, the script displays when the first measurements will be made. Recall that measurements are synchronized with the measurement interval. Since one minute was selected, we wait for the seconds to be 00. Next, we see the display of several measurements showing the day of the week, date and time, sensor, and the temperature as measured. The measurement results, along with time and sensor, are also displayed on my 16 character by two line Led Display.

After all of the measurements have been made, as stipulated by the value in "Number Of Measurements", the script will stop. If you wish to stop the script prematurely, you can simply issue a Ctrl C or press the switch on the breadboard.

Terminal Window At The End Of The Run

Once the script is terminated, the terminal display looks similar to the figure above. This information is from yet another run. I did not coordinate that aspect of this post very well. You see the last of the measurements followed by a salutation and the reporting of the fact that there were no glitches.

Originally, I had problems with measurement failures where the device file could not be read. I call that a glitch and keep track of the number of glitches for the entire run. My solution to recover from these glitches is to unload and reload the 1-wire modules. For every glitch, I only unload and reload the modules a maximum of three times. If there is no recovery after the tree tries, I stop the script and report a message. This problem has not reoccurred, I have not seen any glitches for a long time.

The start time and stop time, are followed by long numbers. These numbers are the number of seconds since January 1, 1970. This is how RDTool records numbers in the .rrd files. If you want the time on the graph to be correct, you must apply these long numbers to the measurements. I report those numbers at the end here in case you wish to investigate the .rrd files. The information in the .rrd files look like gibberish if viewed with a text editor. You have to issue a RRDtool command to see the contents of these files. For more information see my post "RRDtool For Dummies Like Me" under the topic "What Measurement Values Go Into the Database".

Let's look at the code. The code is divided into two scripts, one for the GUI and the other for the main program that makes, graphs, and records the temperature measurements. The reason for a separate script for the GUI was to minimize changes to the main script. The main script is derived from the code shown in my blog entry, "Graphing Real Temperature Data Using RRDtool and PyRRD". The code for the GUI was developed independently, and has test code at the bottom so it can be run by itself, to check its operation. The GUI script was made a callable module by saving it as a .pyc file. The first 425 lines of the GUI code becomes a single function, guiwindow(). All of the parameters collected in the window are passed to the main script by line 103 of the main script:

     variable_list = guiwindow()

Code For the GUI Window:

I know it's a rather long script (another reason for making it separate from the main script), but a lot goes on here. Each element of the window (widgets) must be defined. Some of the widgets, when clicked on by the user, spurn actions. These actions must be defined. A lot of code is devoted to making the appearance correct. Placement of widgets is somewhat of a challenge. As in most GUI applications, user inputs are checked to make sure required parameters are not missed, or errors made. The operator is made aware of these errors by messages so he knows what to correct. The error checking requires a lot of code. Just look at the function proceed(). Most of the widgets have error messages included in their definations.

The development of the GUI is done using a module called Tkinter. It is probably available for all, common, Linux distributions for the Pi. I have developed all of my code using Python 2.7 so my line to import Tkinter is: "from Tkinter import *". In Python 3.x, that line would be "from tkinter import *". Tkinter is not just for Python, and not just for Linux.

So, how does one get started developing GUIs for their Raspberry Pi projects? There are many references, including "The Python Library Reference". This document points you to other sources, including the Python Wiki, a source with even more sources. One source you need to have available, constantly, is "Tkinter 8.5 Reference: a GUI for Python" from the Computer Center of New Mexico Tech. You need this for no other reason then getting the syntax correct. It is really a great asset to GUI development. Another good source of help is my code above, and code of others, for you to see practical examples. There are even YouTube videos with Tkinter tutorials. There is a ton of stuff out there on the net on Tkinter.

There are a few other items of interest in this script. Note the use of the Subprocess module in the function "getdirectory()". This is a way of issuing the Linux Ls command from Python. Here, Ls is used to see if a directory has been previously made. If it does not exist, the function makes the directory. Another cool thing is what happens, after the user presses "Continue" and the inputs have been found to contain no errors. Before the window closes and operation passed to the main program, all of the user's input gets saved to a file. When the window is opened again, the information in this file populates the widgets. This is done using a module called Pickle (actually cPickle. A version developed using C that runs faster than Pickle). Check it out, Pickle is a very efficient way to do the job and saves you from writing many lines of code.

Main Script For Making, Graphing, and Recording Temperature Measurements:

Since this code for the main script was discussed in my earlier post, I'm not going to say much about it. One element I added is saving the results to a text file. I discussed this earlier in the post. Lines 281 - 288, 312 - 319, and 327 - 334 handle this task. The directory and file name come from the GUI window.

I almost forgot, here are the two graphs:

Graph Corresponding To Window Above

This Graph Is Better For Printing - It Saves Ink

Can A Python Script With GUI Run From Root? Not A Trivial Question. Answer: Yes

Important Preface: While I mention graphing results of measurements in the next paragraph, this post is about the Graphical User Interface. Even though graphing and Graphical contain the word graph, this post concerns a problem related to the Graphical User's Interface and not any problem graphing results.

My last blog post is about my python program that measures temperature and then graphs the results. When that script runs, the user is faced with many questions to answer. All that typing gets quite tedious after a while so I thought about including a GUI (Graphical User's Interface) to deal with the questions. I started slow and included a pop-up window for the user to select a file name. I used the Tkinter module, included in python 2.7. Seemed simple enough. Of course, due to the RPi GPIO code, the script has to be run as root (prefacing the command with sudo). It didn't work!

I got an error: “Client is not authorized to connect to Server…..”. What!!! This error really floored me. I posted a question on the Stackoverflow forum and was told that I should use gksudo or its cousin gksu instead of sudo (gksudo is related to gksu as sudo is related to su).

What is gksudo/gksu? The Linux manual page says: "Their primary purpose is to run graphical commands that need root without the need to run an X terminal emulator and using su directly." The first part of that sentence sounded to be just what I needed. So, I played around with a simple script and ran it with gksudo. It worked fine. Then, I tried more complex scripts and I noticed they almost worked. Finally, I tried my large temperature measuring program run with gksudo (no GUI code in this script), and IT DID NOT WORK AT ALL.

Now, what? Every time I ran a script using gksudo (or gksu) I received this error message: "Xlib: extension "RANDR" missing on display "1.0". I had no idea what that meant but decided I had to find out how to get around it. Consequently, I did a search on RANDR. I found nothing to explain what RANDR is but I found quite a few forum posts where people asked about the exact same error message. This message pops up in a diverse universe of Linux applications run on a variety of platforms. There were a lot of answers, too. However, none had a solution I could use. Many of the answers said to ignore the error message. In my case, I knew I could not ignore the error - my script fails.

I next posted the question on the Raspberry Pi forum and the Stackoverflow forum. I explained about getting the no permission error with sudo, and the RANDR error with gksudo and the fact that my script fails. My responses on both forums was no response at all.

Finally, my son helped me out by pointing me to a forum post he found that gave me the answer that worked:

To the bottom of this file:

          pi@raspberrypi: ~/.bashrc

Add (first line optional):

          # allows X11 graphics to be run from root

          xhost +

The first line, of course is just a comment. What is xhost + doing here, and why?

The Linux manual page says that xhost is used to add and delete host or user names to the list allowed to make connections to the X server. By saying xhost +, we give every Tom, Dick, Harry, and Edward Snowden permission to connect to the X server. This includes root.

And, what is X server? From the Linux Information Project, "An X server is a program in the X Window System that runs on local machines (i.e., the computers used directly by users) and handles all access to the graphics cards, display screens and input devices (typically a keyboard and mouse) on those computers." The X window system used by the Pi's Linux distribution is X11.

Since we give everybody permission to make connections to the X server (xhost + does that), are we putting our Raspberry Pi in danger? If you have other users that have access to your Pi, they may be able to pop a window into your display space. But, if you are the only user there should be no danger of applying xhost +

Graphing Real Temperature Data Using RRDtool and PyRRD

My previous blog post, "RRDtool for Dummies Like Me" was a tutorial on the use of RRDtool and PyRRD to graph data from a python script. For simplicity, I used made-up data. This post talks about plotting real data taken with the Raspberry Pi.

In June and July of this year, I wrote five blog posts about measuring temperature with the Dallas Semiconductor DS18B20 One-wire temperature sensor. The temperature measurements were reported to the terminal screen and my 16x2 LCD display (seen in photo in blog header) so would not be saved. I did modify the script to send the results to a file but have not reported on that effort. Perhaps that would make a worthwhile post for later.

It just seemed natural to want to plot that data to see the change of temperature with time. That led to my investigation of RRDtool and PyRRD, and to the tutorial in my last post. This post combines temperature measurements with plotting in a python script.

I have two DS18uB20 temperature probes in my system. One looks like a small transistor and sits on the breadboard and the other is at the end of a cable. For the graph below, I put the cable sensor in a glass of hot tap water while the breadboard sensor measured ambient air temperature.

Plot With Real Temperature Data

The python code follows.

The script is an evolution of code I reported on previously. All the One-wire temperature code was discussed in my June and July posts. The plotting code grew out of the code in the last post. The code for the LCD display came from my posts of March and April of this year.

My python script may seem pretty large, but I wanted to make it useful, and to incorporate all of the hardware connected to the Pi that makes sense to use. These are some of the features:

1. Choose to use either sensor, breadboard or cable, or both sensors.
2. Choose graph characteristics:
   1. If desired, give the breadboard, and/or cable sensor another name.
   2. Give the graph a title - mandatory
   3. If desired, enter a comment that prints on the graph.
   4. Select graph with either black or white background, or choose to make both graphs. White background is best to print. It's easier on printer ink. I think black looks better on the screen, like as seen in the sample graph.
   5. Enter the height from 100 to 400 pixels. If you are plotting from one sensor of a temperature that does not change much it is better to choose a lower value for the height. Otherwise, RRDtool may repeat the values on the Y axis, like: 70, 70, 70, 71, 71, 71, 72, 72, 72, etc.
   6. Choose a file name. Up to four files will be created with that name: an .rrd file for each sensor chosen, and a .png file for each background color chosen.
3. Choose the maximum number of measurements to make for each sensor. The script will stop once that number of measurements (for each sensor) is reached. In the future, I may modify the script so that once the maximum is reached, new measurements replace the oldest measurements. Remember, this is how a round robin database works.
4. Choose the time interval between measurements, in minutes. For each sensor: The first measurement will be taken at the first time corresponding to the time interval. For example, if the time is now 37 minutes past the hour, and 15 minutes is selected as the time interval, the first measurement will be at 45 minutes past the hour. If you select the time interval to be one day, the first measurement will be made at midnight. When making the plot, RRDtool will only place a measurement point at a time corresponding to the nearest measurement interval. Again, as an illustration, if we choose a day as the time interval, and we made a measurement at 8PM, the plot will show the measurement at midnight, and --- unless it is the first measurement, the value will be different. That value is not even the linear interpolation of the measured value. See my past post. Strange, but that is the way the tools work.
5. After entering all of the user information, you have a chance to review and enter the information again, if you are not satisfied.
6. You can choose not to proceed after entering the user information. This will stop the script.
7. As well as plotting the temperature data, the average temperature for for each sensor, is calculated and displayed below the plot.
8. The current measured temperature value, for each sensor, is displayed on the 16 character by 2 line LCD display.
9. All measurements are sent to the terminal along with the time of the measurement.
10. There are three ways to stop the script:
    1. When we have reached the maximum number of measurements.
    2. Pressing CTRL-C from the keyboard (Issuing a keyboard interrupt).
    3. Hitting the switch on the breadboard.

There were two things that did not work out as I wished. The first is that RRDtool/PyRRD makes adding some text to the graph somewhat difficult. The title requires you to escape all spaces and probably some punctuation. A space needs to be "\ " in the variable (title_it). It's so easy to forget to add that backslash that I tried to have the script do that for you. It would seem that the following would work:

      title_it = title_it.replace(" ", "\ ")

It doesn't. It replaces a space with "\\ ". This is a title prints on the graph: This\ is\ a\ title, which looks pretty dumb. The following does not work either, giving the same result:

      title_it = title_it.replace(" ", r"\ ")

Since I had no luck adding the backslashes before the spaces, I have the script, in the function check_title, check that the user added the backslashes. Note that the user type "\ " for each space, but the function looks for "\\" before each space (line 103). That makes sense, the first backslash escapes the second backslash.

Here's the second thing that did now work out: I wanted to handle all of that user input with dialog boxes. Using stdio is a bit clunky and if you decide to change some input, you have to enter everything again. Besides, I wanted to learn to incorporate those features. Python has a nice module called Tkinter which brings the graphic users interface to python and other programming languages.

I experimented with a simple program to bring up a dialog box to enter or choose a file name. It worked just fine. I could run it from a Idle Python shell, or from the stdio. No problem. Then I incorporated some GPIO code which meant I had to run the script as root. Now it failed. It threw an exception saying, "Client is not authorized to connect to the Server .....". By client, I assume it means root. Obviously, if the user is me, it works fine, but not if the user is root.

RRDtool For Dummies Like Me

Introduction

RRDtool is primarily a graphing program. The developers refer to it as database program, and it certainly stores data, but I don't think you will trouble with it unless you want to generate graphs. And, the graphs are rather limited. Use it to graph the results of periodic measurements of such things as temperature (which got my interest), humidity, network data, distance, people coming coming and going, speed, daily stock market prices, etc. After spending a great deal of time with RRDtool, I don't believe you could use it to graph something like robberies against midday temperature, or test scores against hours watching TV. In other words, the X axis should be time.

An RRDtool Graph

I am blogging about this because I find this tool useful. However, after reading tutorials, reading usage documentation, and reading blogs of others making graphs with RRDtool, I found it difficult to put it all together and get satisfactory results. There are many idiosyncrasies. unexpected results, and pitfalls along the way. I think I mastered the tool, but only after many hours of toil. I want to share what I learned with the Raspberry Pi community.

RRDtool is widely used in industry, but mainly, it seems, by IT departments to track network data and memory usage. As I discuss this tool, it will be apparent that it was designed for that purpose.

Round Robin Databases

RRD in RRDtool stands for Round Robin Database. It's a different kind of database, there are none of the traditional database elements like fields and records. You store time and data. Just numbers. Those numbers relating to time, are weird. If I take a measurement now (September 3, 2013, 4:53:19 PM), the time in the database would be 1378241580.

So what's a Round Robin Database, and why use one? The purpose of this type of data storage is to limit the amount of data stored so your database does not keep growing on you (beyond a certain point). As far as I can see, that is its only advantage. You tell it how much data to store, and when that number is exceeded by new data, the oldest data is replaced by the new data. The "Round" in "Round Robin" gives a visual representation of the process. Picture something round, like a Ferris Wheel, for example. The wheel has a limited number of cars. Let's say we put one piece of data in the first car and the wheel turns to the next car. The next piece of data goes into the second car and the wheel turns again. This process is continued. Eventually, the first car comes around again. Now, if there is new data, it replaces the data already in the car.

Take a look at the sample graph. If it were me doing the plotting, I would connect two adjacent data points with one line that slants from one point to the next. RRDtool does not plot that way, there is a series of horizontal lines from one time interval to the next, then vertical lines from one measurement value to the next. It looks weird if you have few data points like I do in the example. If you have many, many data points, the horizontal lines become very short and the graph looks better.

RRDtool For Python - PyRRD

RRDtool is available for many platforms. For the Pi and Linux, you take the apt-get route (sudo apt-get install rrdtool) . RRDtool is a command line program. While you can write shell scripts to make your work reusable, I wanted to run it within my python programs. Of course, you can issue command line commands within python, but it becomes very cumbersome. There is a good solution, and that is a program called PyRRD 0.1.0 which is a python interface to RRDtool. It seems to have all the functionality of RRDtool.

There is one function of PyRRD that does not work and that is the function called Fetch. This very useful function is used to retrieve the data you have in the database. I have used it frequently as you will see when I show the data in a database. However, I don't use it from PyRRD, but from the command line from RRDtool. Be sure to download both programs. There are many forum posts about this Fetch problem dating back to 2009, but there has not been a solution. It's no big deal issuing fetch from the command line.

PyRRD 0.1.0 is not in the Debian depository so you can't use apt-get command to get the program. You must download it and install it. Go to the PyRRD website for the download file and instructions. If it fails saying it can not find setuptools (not installed with Raspbian) execute the following: wget https://bootstrap.pypa.io/ez_setup.py -O - | sudo python

I spent a great deal of time with the sample program under the website topic "Usage". I wanted to understand every line. After typing commands into Idle's python shell, I decided I needed something reusable so made a python script out of the code. That way I could experiment by making changes and seeing the consequences. If you wish to do the same, change the lines starting with vdef1= and vdef2=. They should end in cdef1.vname not def1.vname.

I followed the instructions in the section labeled "Python Bindings". I could not get this to function. Apparently I'm missing a library . However, things work fine without it.

Documentation You Absolutely Need

In addition to the PyRRD website, please see the RRDtool website. In particular see the tutorial by Alex van den Bogaerdt. Also, see the documentation for the rrdcreate function, and any other function of interest.

Once you have PyRRD installed you need to know the functions available to you and what parameters can be passed into those functions. This information can be gleaned from two python files. With the Debian distribution those files are in:/usr/local/lib/python2.7/dist-packages/PyRRD-0.1.0-py2.7.egg/pyrrd. The file rrd.py has about everything you need for logging data. graph.rrd has everything about graphing. You will refer to these two files constantly.

The Code To Produce the Graph Above

Let's jump in by showing the code that produced the sample graph in the beginning of this post. We will go through this in detail:

Sample Code For Graph Above

Let's Talk About Time

Remember, back in the introduction, I mentioned the time in the database looks like 1378241580. What's that all about? That number represents the number of seconds elapsed between January 1, 1970 and September 3, 2013, 4:53:19 PM UTC.

In practice, before we can deal with time, we must decide on the time interval, in seconds, between measurements. In my example, I have chosen every minute or every 60 seconds. The data shown in lines 56 through 87 show every measurement reported at a "perfect" 60 second interval. This means if you take the time of any measurement and divide by 60, you will have no remainder. Later, I will discuss what happens if you deviate from this by shortening or extending the interval.

Code lines 16, 17, and 18 compute the seconds since that magic date. Line 16 gives the UTC time, of right now. Line 17 computes the total number of seconds, including a decimal component, since Jan. 1, 1970. Line 18 rounds off the number of seconds to the nearest minute. If we were taking measurements every 5 minutes, line 18 would be: secondssince = int((secondssince + 150.5)/300)*300. Lines 20 and 21 are useful for fetching the data back out of the database. We will see this later.

Creating the Databases

Before we add any data we have to create the structures for the databases. Since our sample graph has two sensors we will create two distinct databases. This happens in lines 23 to 51 of the code. We'll look closely at the first sensor database (lines 23 to 37). We assemble all of the information necessary for the database and apply the RRD function in line 36. The RRD function needs a lot of information and we build that information in lines 27 through 34. Line 37 creates the database.

First, we create a list using the DataSource function. It will contain the name we wish to use for this database (Sensor1), the database type (GAUGE), and the heartbeat (90 seconds).

There are five types of data source types: GAUGE, COUNTER, DERIVE, ABSOLUTE, and COMPUTE:

GAUGE is for data like temperature that could be measured by a gauge. The data is taken as is.

COUNTER is for data taken from an ever increasing counter like an odometer. Of course, overflow has to be taken into account, as when an odometer rolls over. Apparently, this used in the network business. You supply a counter reading but data goes into the database as a rate, not the counter reading itself.

DERIVE, and ABSOLUTE are variations of COUNTER. See the rrdcreate documentation on the RRDtool website.

COMPUTE is a type I have not investigated. Again, see the rrdcreate documentation.

Heartbeat is the maximum time, in seconds, after a data point is reported, that is allowed to pass before a result is considered "UNKNOWN". For example, if a measurement is taken now, and the heartbeat is 90 seconds, if another measurement is not reported for more than 90 seconds, the next value is reported as "UNKNOWN". If you are averaging results (more on this later), a zero value will not be averaged into the result. If you were averaging three values and one was unknown, you will only average two values.

Next we will set up one or more Round Robin Archives. This controls, along with the data source type, what values are actually stored in the database. Each archive becomes an element in a list. I have two archives for each sensor so the list will have two elements. See lines 33 and 34 for sensor 1. Details of the Round Robin Archives follows.

If you take measurements every minute for one day, you probably would not wish to graph each value (1440 data points), so you might wish to consolidate the data. Perhaps you will average each hours worth and graph 24 points, one for each hour in the day. Or you might like to take the lowest, or highest value of the 60 values measured during each hour.

The first and third parameters of the RRA function handle how you wish to consolidate the data. The third parameter is "steps". This is the number of measurements you will take before you enter a value into the database. In my line 33, I am going to take two measurements then apply a consolidation function. In this case, it is AVERAGE, which is the "cf" parameter. "cf" is the first of the RRA function parameters. I take two measurements, take the average of the two and only report the average in the database.

There are four consolidation functions available. After taking the number of measurements as called for in the "steps" parameter, you:

LAST: Take the last value of those measurements.

AVERAGE: Take the average value of those measurements.

MAX: Take the largest value of those measurements.

MIN: Take the smallest value of those measurements.

In my line 34, I establish my second Round Robin Archive, which shows "steps" of 1. Therefore, every measurement will be recorded. In this case, all of the consolidation functions will give the same result. My sample graph uses this Round Robin Archive, not the one where I average two measurements.

The parameter "rows" controls the number of values you can record before filling the database. My first archive fills in 48 minutes because one value is recorded for two measurements, the second fills in 24 minutes. Further data overwrites the oldest data. I'm not too sure about the xff parameter. All of the examples I have seen show a value of 0.5. It is associated with the unknown values. The values from both archives will go into the same database file. You can have many round robin archives and they all go into the same database file.

We have reached the line 36 where we assemble the information we have generated using the RRD function. We also add a few items. First, is the file name for the database ending with the rrd extension. We also add a start time. It is important for the start time to be at least one measurement interval before you add data to the database. In this case the measurement interval is 60 seconds. If you don't do this, you may lose the first measurement value.

Sensor 1 and sensor 2, so far have the same information except each has its own database file name. Finally, we issue the command to create the database is line 37 (line 51 for sensor 2). The parameter debug is optional with the default "False". If you make it "True", all the data is spewed out to the stdio, for you to check.

Adding Sensor Data

Lines 56 through 87 illustrate how we would report the measurement data. The data here, which represents temperature measurements from two sensors, is entirely contrived. We attach the BufferValue method to each of the two sensors, variables sen1, and sen2. The format of the BufferValue parameters is "time, measurement". After all the data is collected, we the use the update method to get the measurements into the databases (lines 89 and 90).

What Measurement Values Go Into the Database

You would think that the measurement values in the database would be the same as the measurements you report to the database. That is not always the case. Before I created the sample graph code, I was playing around with measurements at different times. I was using a "step" size of 5 minutes or 300 seconds. I added some data and from the command line I used the fetch command to see exactly what wound up in the database. I originally started with a time of 1000000000. I was surprised to see only the first data point matched what I put in. I found the next time that was an even multiple of 300 was 1000000200. Now, if all my measurements were multiples 300, the values in the database matched what I put in.

The following show data into the database and what wound up in the database. I used the rrdtool fetch command to retrieve the database data. As you recall, this command did not work within PyRRD. The command was issued from the command line as:

          rrdtool fetch filename.rrd LAST --start 999999700 --end 1000004500

Data Into Database

Data In Database

As you can see the data in and out match.

I also had two Round Robin Archives. What you see above has "steps" as 1, and consolidation function as LAST. I also configured one with "steps" as 2, and consolidation function as AVERAGE. To get access to the Average archive I simply substituted AVERAGE for LAST in the fetch command. Here is the results:

Consolidation: Average, Steps: 2

You would have thought the first value in the database would have been the average of the first two measurements. But, no, it is the first measurement. The next value in the database is truly the average of the second and third measurements. This pattern is continued until the end. The last value is the average of the two measurements 66.7 and 68.2. The last measurement of 70.4 is lost. It seems that whatever you do to the data, and the measurement times, the first value in the database is always going to the the first measurement.

What happens if you put in more data than is in the "rows" parameter. If "rows" is specified as 12, let's add 18 measurements and see what happens.

Eighteen Measurements When Rows = 12

Result of 18 Measurements

As you can see, the first six values are wiped out and the database still has 12 values.

So far we have looked at "perfect data". Let's look at what happens when the data is sent to the database not on the exact time interval. The following code sends measurements 100 seconds after the 300 second interval:

Data Sent 100 Seconds Late

Results of 100 Seconds Late

We notice three things: One, the times in the database do not equal the times that we sent to the database. The times in the database are still at the 300 seconds, exact, interval. Second, the first value in the database, 72.4 matches the first value sent to the database. Third, none of the other eleven values in the database match the values sent to the database.

Let's try to analyze how RRDtool decides what values to put into the database. The next code, sends measurements at times either on, before, or after the exact interval. There is one measurement that falls right on the heartbeat definition, and one measurement after the heartbeat. To make it easier to generate a visual representation, I am making the "steps" 10 seconds, and the heartbeat 15 seconds.

Data Sent at Variable Times

Results For Variable Times

Computing the Values In the Database

Resulting Graph

I would expect a linear interpolation to be used to determine the values in the database. For example, the value at time 1000000010 to be determined by drawing a line between the value at 1000000007 and 1000000022 and seeing where it crosses the 1000000010 line. That value would have been 3/15*(69.7-71.8)+71.8=71.38. The value in the database is 70.495 not 71.38, so it is obvious linear interpolation is not used by RRDtool.

In my graphic "Computing the Values In the Database", I indicated where the value in the database (black numbers) come from, but not the rational for why the various mechanisms for computation were used. Are the rules and algorithms those for IT? I'm not a mathematician or IT person so I don't know. Please feel free to supply a little insight here. I won't analyze these results further but would advise you to report all measurements at the times that exactly match the chosen "perfect" intervals. If a measurement is missed it would be better to repeat the previous measurement than to report nothing.

Graphing The Results

OK We're ready to discuss the graphing - the fun part. One word to mention up front is that all of the functions to be discussed have many optional parameters. I'll discuss those I think to be the most important, but by no means all. The function graph has one mandatory parameter and 35 optional ones with default values.

DEF, CDEF, and VDEF Functions

Of these three functions only DEF is mandatory. I used DEF and VDEF for the sample graph.

DEF means Data Definition. This function links the database file and the Round Robin Archives to the graph. It takes takes four parameters:

rrdfile: The file name, with its path, of the database file.

vname: An arbitrary name. You can use this name in later definitions. This name will become equal to the file name in rrdfile.

dsName: Data Source name. The same dsName you specified when setting up the database. Line 38 for sensor 1 in my code.

cdef: This is not the same as the function CDEF. It has the default value of "AVERAGE". I will be graphing the "LAST" values from my Round Robin Archives so I must specifically specify "LAST". The value in dsName will become equal to the value in cdef.

CDEF means Calculation Definition. CDEF works on each data value in your database. I have not used CDEF in the code above but I will show an example where I convert the values from Fahrenheit to Centigrade. You can also use CDEF to make decisions. I'll show an example where I use it to plot different colors if the temperature is above or below the average temperature. Neat stuff. CDEF takes two parameters: vname which is an arbitrary name you can use later to refer to the calculation defination, and rpn, the calculation formula.

All of the calculations in CDEF and VDEF are done using Reverse Polish Notation. That is what the rpn stands for. When HP sold scientific calculators using rpn, I avoided purchasing one just so I would not have to learn Reverse Polish Notation. Now, all these later, I have to learn it. Good Grief! Anyway, it's not too bad. There is an excellent tutorial on the RRDtool website

VDEF means Variable Definition. This is used to calculate a single value based upon all of the data you are using from the database. My graph uses it do draw the continuous lines that represent the average of the database data values. I also use to print those values below the graph.

VDEF has two parameters, like CDEF: vname and rpn. The VDEF calculations in rpn are limited to LAST, AVERAGE, MAXIMUM, MINIMUM, and PERCENT.

LINE, AREA, and TICK Functions

LINE obviously draw lines on the graph. For each sensor, I have two line definitions, one for the data points (lines 99 and 107), and the other for the average (lines 102 and 110). The LINE function has six parameters, all of which have default values:

width: Width of the line. You can specify a floating number. I don't know if this is in pixels or what.

value: Use this if you wish to draw a continuous line at some value parallel to the X axis.

defObj: This important parameter controls what data you draw. Use the variable name derived from the DEF, CDEF, or VDEF functions. I use the DEF choice to print the data, and the VDEF choice to print the average line. If I were to do some calculation on the data using CDEF, I would use the CDEF choice. You will see this later.

color: If you don't specify a color, the line will be invisible. Color is specified using Red, Green, Blue or RGB. The first two hexadecimal digits are red, the next two are green, and the last two are blue. 00 means the color is missing, FF means the maximum value of that color. My line in code line 99 is green.

legend: Under the graph itself, you see four colored squares with some text. The text is what is specified in legend

stack: If you make stack=TRUE, then the values in this line will be added to the values in the previous line. I don't know when you would use this.

AREA draws a solid colored area from the X axis to the database value. It takes the same parameters as LINE. My sample graph does not AREA, but I will discuss it later.

TICK: This one is strange to me. I'm just going to copy the documentation in the function definition:

Plot a tick mark (a vertical line) for each value of vname that is non-zero and not *UNKNOWN*. The fraction argument specifies the length of the tick mark as a fraction of the y-axis; the default value is 0.1 (10% of the axis). Note that the color specification is not optional. I'll show an example of this. It's not worth mentioning further.

GPRINTS and COMMENTS

GPRINT or Graph Print. The next line up from the last line on the sample graph show the usage of two GPRINT statements. This is useful for printing some variable along with text. The variable in this case is the average value of the temperature measurements for each sensor. GPRINT takes two parameters:

defObj: This tells where the variable number comes from. In the case of my sample graph, the values come from the VDEF definitions.

format: This parameter contains the text you want to print on the graph including a formatted reference to the value in defObj. In my graph, this is the "%6.2lf". Please note the character between the "2" and the "f" is not a "1" (one). It is an "l" (el). This means we will print a value with two decimal places that will take up at least 6 total character places.

COMMENT: The last line on the graph is a comment. The function has two parameters, the text itself and autoNewline. autoNewline has a default value of TRUE. I haven't tried making this FALSE.

Some characters have to be escaped with the "\" character. According to the function definition the ":" character must be escaped, I don't know what other characters. The space character does not have to be escaped here. In other parts of the graph, the space character does have to be escaped. I used the escape character to start the comment with a tab ("\t").

Color Attributes

This useful but optional element really jazzes up the graph. This allows you to control the color of nine elements:

back: Background. The area surrounding the graph's grid. Solid black in my example.

canvas: Canvas. The color behind the graph's grid. Dark gray in my example.

shadea: Left, top border. A line along the top and left side bordering the graph. Too light to see in my example.

shadeb: Right, bottom border. A line along the bottom and right side bordering the graph. Too light to see in my example.

mgrid: Major Grid. Color of major grid lines, minor grid lines are darker. Light gray in my example.

axis: X and Y axis. White in my example.

frame: I don't know what this is.

font: Font color. White in my example.

arrow: Color of the arrows at the ends of the X and Y axis. White in my example.

Finishing Up

After the import command, code line 136, we give the path and name of the file that will hold the graph. This will have a .png extension.

Next, we establish a variable, "g" in the case of our sample graph, and pass it more information. The "Graph" function has only one parameter you must supply, and that is the name of the graph file. In addition, it has 35 parameters with default values, so you only have to assert the ones you wish to change. I have included seven of the 35 here. Note the "start" time is one minute before the first measurement is reported. This assures the first measurement will be plotted. Notice also that the space must be escaped in the "vertical_label" and in the "title". Without the escape you get an error if you include the space. Before I figured out the "\ ". I replaced the space with the underscore character. Some of my graphs that follow will have the underscore.

Next we pass all of the variables we established earlier, the variables made from the DEF, CDEF, VDEF, LINE, AREA, GPRINT, COM, etc. We make a list of all of these variables and send them to the "g" variable. See lines 140, 141, and 142.

The order of the items in the list is important. It establishes the order of what is drawn on the graph. The GPRINT variables must follow any line, area. or tick because they are included after the plotting. The comments must be the final item in the list because comments are included at the very bottom. If you put GPRINT or COM variables earlier in the list you will throw an exception.

Finally, we issue the g.write() command and the graph is made.

Controlling The Graph

There are many items you can not control. You can't place a comment at the top, for example. Theoretically, it is possible to control the start and stop values of the X and Y axis. There is a GraphXGrid and a GraphYGrid function. The GraphXGrid function documentation basically said it is difficult to change the grid.

The number of grid lines, both major and minor are determined by the data and the size of the graph. Change the width and height in my code line 138 and you will get more or less gradations. The positioning of the legends under the graph are controlled by RRDtool. You do control the order by the order of the items in the list you provide to the variable "g" (in the case of my code).

You don't control how the time below the X axis is displayed. My sample graph shows hours, in 24 hour format, followed by minutes. If you data is taken over many days, or months, or years, RDDtool will adjust accordingly to its rules.

More on CDEF

Temperatures in Centigrade

The graph above shows the temperature now plotted in Centigrade Rather than Fahrenheit. Most of the changes are shown in the code following the graph. For brevity, I've only shown the changes to sensor 1.

Line 2 is new. It uses the CDEF function to apply a calculation to each temperature value in the database. The calculation is defined in the "rpn" parameter. The calculation is the formula C = 5/9*(F-32), but in Reverse Polish Notion. We do this rpn calculation from left to right. For each database value, we put put values on the stack until we reach an operator sign. Therefore, the database value and the number 32 go on the stack. When the "-" operator is reached, two values are removed from the stack (in this case emptying the stack) and subtracted from each other. That resulting value is put on the stack along with the number 5. When the "*" sign is reached, those two numbers are removed from the stack and multiplied. The result and 9 go onto the stack and removed when "/" is reached. The last result is divided by the 9 to produce the final result.

Notice, in line 3, that plots the line of data, defObj now refers sen1_cdef not sen1_def. Likewise, in line 5, rpn='%s,AVERAGE' % sen1_def.vname, is replaced by rpn='%s,AVERAGE' % sen1_cdef.vname

Printing Areas Instead of Lines - Bad Example

Areas In Place of Lines - Useless Application

You can replace the lines representing the database values with solid, colored, areas as shown in the plot above. I changed some of the database values for sensor 2 so some of them will be less than the values of sensor 1 at the same measurement time. I also made sure I plotted sensor 2 before sensor 1. This was done here:

g.data.extend([sen2_def1, sen2_area, sen2_aver, sen2aver_line])

g.data.extend([sen1_def1, sen1_area, sen1_aver, sen1aver_line])

As you can see, the second area plotted can cover some of the plot of the first area plotted. If I had plotted sensor 1 first, the green area would only be visible for four measurements. As it is, since I reduced some of the sensor 2 values, below sensor 1 values, four of the sensor 2 values are lost. This is not a very good use of areas.

Printing Areas Instead of Lines - Good Example

Areas In Place of Lines - Useful Application

The graph above illustrates plotting useful solid areas. And, it illustrates the use of the IF statement to make decisions, in Reverse Polish Notation. For this graph, we are only printing from sensor 1. We will print the area in green if the database value is below 66 degrees. If the values are above, we print the area in red. The important code changes are shown below the graph.

Only code lines 9,10, 16, and 17 are new. Lines 9 and 10 use the CDEF function to do calculations while lines 16 and 17 use these calculations to define the two areas. We will look at the calculations in the "rpn" parameters. Here we are making decisions and applying those decisions to the data.

Let's look at "66,Sen1data,GT,Sen1data,0,IF" and see what is going on. For every point in the database we compare the database value (in Sen1data) to the number 66. If 66 is greater than the database value, the result is 1, otherwise, 0. Next, finding the "IF" at the end, it says that if the value is 1, the result of the calculation is the database value, otherwise it is 0. This result is passed to the AREA definition in line 16, which defines a green area on the graph. You can see that only between 11:50 and 11:59 will the values be below 66. The second area that plots a red uses the calculation in line 10. Here the calculation is "66,Sen1data,GT,0,Sen1data,IF". In this case, again, if 66 is greater than the database value, the result is 1, otherwise 0. Now, however, if the value is 1, the final result is 0. If the result 1s 0, the final result is the database value. This calculation results in the red area defined in line 17.

If the value is exactly 66, it will print green. If I wanted to have it print red, I would use LE instead of GT and rearrange things. My real intention was not to use a discrete value like 66. I wanted to use, instead, the average value, which is calculated in line 8. My attempt to do that is what is in the commented out lines 12 and 13. I played around with this for some time, but everything I tried resulted in an exception. Unless, I did not hit on the correct solution, I can only conclude that you can only use one variable in the "rpn" calculation. Too bad!

Finally, I really wish I could have changed the Y axis and have it start from some other value besides 0. I tried replacing the 0 in the rpn expressions with 50. The final plot was a mess. RRDtool decided not to start the graph at 50, but at 40 instead. What I got was a plot that was red from time 0 to time 11:50. From 11:50 to 11:59, the plot was red from 40 degrees to 50 degrees and green from 50 degrees to the database value, then red to the end of the plot. I don't understand this at all.

That's All Folks

Hope this is worthwhile. Happy plotting.

Back in June and July I presented a series of posts titled, "A Raspberry Pi Thermometer". My next project will be to update that project useing RRDtool and PyRRD to plot the temperature readings. Stay tuned.

Gertboard - Camera Remote Control - Arduino Sketch #2 - Motion Detector

Continued from last post

This final version of the camera remote control project integrates the motion detector and removes the user input. This motion detector is properly called a PIR (Passive Infrared) sensor. It detects heat given off by bodies around it. The sensor is actually split in half. Motion is detected by there being a slight difference in the heat measured by the two halves. An IC mounted on the printed circuit board processes the sensor signals and gives a digital output that we pass to the microcontroller on the Gertboard.

Here is a very informative link about these sensors My sensor is slightly different than the one in that last link. Mine looks exactly like the one you see on Adafruit's web page. It has the advantage of two potentiometers, one to set the sensitivity, the other to set a time delay between the time motion is detected, and the time the output goes to a logic high. I spent a bit of time adjusting the sensitivity. The signal from the motion detector has an LED in the path on the Gertboard (see the block diagram in the introductory post of this project). This LED gives feedback that makes it easier to set the sensitivity.

Here is the new code:

I don't think it's necessary to discuss the code further, we covered it pretty well in the last post. The changes should be pretty easy to understand.

I would like to mention that this version does not require the Gertboard, or the Raspberry PI. Once the ATmega device is programmed, it will not lose the program even if it is lifted off the Gertboard and placed on another PCB. Everything can be bundled on a small printed circuit board like Adafruit's Perma-Proto Half-sized Breadboard PCB and placed in a small box with the motion sensor and the RF transmitter. Connect 5V and away you go.

Gertboard - Camera Remote Control - Arduino Sketch #1

Continued from last post

I have covered the project scope, the hardware involved, and the digital patterns we have to reproduce. What's left is the code to make it all happen. I have two sketches, Arduino parlance for source code. The first was written without using the motion detector. Instead, the user is prompted to input an "a" or "b" at the keyboard. An "a" transmits the code to take a picture. Pressing "b" is like pushing the camera exposure button halfway down. The second sketch, presented in the next post, incorporates the motion detector and eliminates the keyboard input.

I am using the Arduino IDE (Integrated Development Environment) to write the sketch, compile the sketch, verify the C code, and upload the machine code to the microcontroller. The only IDE function we cannot use is its serial monitor which provides screen output and keyboard input. The IDE's serial monitor requires a USB connection (as would be used with an actual Arduino product). The Gertboard has no USB. If we wish to send and receive data from the microcontroller, we need an alternative.

That alternative is a terminal program called Minicom. It's like the old DOS Telex program used to talk over the modems we used in the old days. The Gertboard User Manual talks us through its installation and configuration. Minicom communicates through the UART(Universal Asynchronous Receiver/Transmitter) port. This two pin port is signified by the pins TX and RX, both on the microcontroller and the Pi/Gertboard. See the block diagram two posts ago. TX of the Pi is connected to RX of the microconroller, while RX of the Pi connects to TX of the microcontroller. The stdio program we usually use for the Raspberry Pi is LXTerminal. Even though it has "Terminal" in its name it's really a command line program. If fact, as we do with most programs, we launch Minicom from LXTerminal.

Arduino IDE and Minicom on Pi desktop

Let's take a look at the script:

Thanks to the Arduino IDE, writing the script (C code) and getting the code onto the microcontroller is simplified. The functions for serial communications, time delays, and handling digital and analog inputs and outputs to the ATmega device are available without adding libraries. There are not even any #include statements in my script. Once you write the script, you don't worry about makefiles. One click handles compiling the code and uploading the machine code to the microcontroller. All of the built-in functions as well as basic C language is included in a very useful help reference. There are other libraries included in the software, as well, that you can include in your scripts. For example, there is a library for the 16x2 LCD displays, and a 1 Wire library (see my temperature sensor posts).

My script is divided into three parts: variable assignments, functions I have written, and the main part of the program. The main part of the program is divided into two parts: a "setup:, and "loop", as required by the IDE. "setup" runs when you first apply power, or first upload the code. It establishes the UART baud rate at 9600 bps and makes the ATmega pin we connect to the RF transmitter an output. "loop" runs continuously after that.

Looking at the variable assignments, there is something I do not understand. You see the variable "one_pulse", it establishes the minimum pulse width in microseconds. This is the time for the "H" or "L" pulse (see below), the time for all other characters are multiples of the time for "H" and "L". From the Audacity display, using the camera remote control transmitter, I calculated the time for "one_pulse" to be 284 microseconds. The time in my script, after altering "one_pulse" to match the patterns, is 415 microseconds, a difference of 46%. I've done a bunch of testing, and it's a mystery to me. If anyone has an idea please let me know.

The heart of the script is the two functions, "send_bits" and "transmit". As we cycle through the pattern we send the characters to "send_bits" then "send_bits" passes the definition of each character to "transmit". "transmit" sends the pattern to the microcontroller. I've repeated the diagram from the last post here to make it clearer how the characters, "H", "L", "E", "A", "B", and "T" are defined:

For clarity: Only the Beginning of the Pattern Is Shown

An important note: The 434 MHz transmitter inverts the bit logic. If I send a low pulse from the microcontroller, to the RF transmitter, the RF transmitter sends a high pulse. That is why a low is written to outpin when the variable "is_high" is true. I think the rest of the code should be pretty straight forward.

Gertboard - Camera Remote Control - Finding The Patterns

Canon Camera Remote Control

Continued from last post

There are two common types of remote controls. The remote control for your TV, DVD player, and audio system are generally infrared devices. They are good for short range, line of sight, applications. There is not much that can interfere with their signal, but they must be aimed to function. My camera remote control is a radio frequency device, which does not have to be aimed, operates over a longer distance, and can communicate through walls. Generally, there are two carrier frequencies (the constant, high frequency, that carries the useful information) in use for these devices, 315MHz and 433MHz. My remote uses 433MHz (I see references to both 433MHz and 434MHz for the same hardware - I use both freely in my blogs because I can't keep track of which number I used last).

My project really comes down to replacing that transmitter you see in the photo with my Pi/Gertboard/Atmega. To do that I have to eavesdrop on the transmitter when it talks to the receiver. I did not develop the means to do that, the technique was passed down to me by my son on his blog. You can see his post here. My setup, and a diagram of the connections follows:

Receiver Hardware

Receiver Connections

The 433MHz receiver, from Sparkfun, is the green rectangular printed circuit board sitting on the breadboard. The black square thing hanging off the breadboard is the 3.5mm stereo jack. It can be purchased from Sparkfun. The jack's pins would not fit into the holes of the breadboard so I soldered two wires to the jack's pins and plugged those two wires into the breadboard. Looking at the bottom of the jack with the four pins to the back (like in the photograph) the left most pin connects to the 1Mohm resistor. I soldered a wire across the ground connection at the front to connect to the circuit ground. That connection is fairly obvious from the photo.

I added the capacitor from habit of adding bypass capacitors to circuits whether they need them or not. It's purpose is filter out any garbage on the 5V power connection to the receiver. It's probably not doing much so if you don't have one don't bother about it.

The receiver requires 5V plus or minus 0.1V. My 5V is supplied from a 5.1V, 0.7A, cell phone charger I happened to have lying around. The charger has a USB connector so I used Sparkfun's Hydra cable to connect to the binding posts of the breadboard assembly. It has a USB on one end a several connectors on the other end. I used the cable's alligator clips to clip onto the binding posts of my breadboard.

Be very careful with the Hydra cable assembly. It would be very easy for the two alligator clips to short to each other, or for the positive 5V alligator clip to short to the sleeve of the barrel connector, thus shorting out your power source. Always connect the alligator clips to the circuit before applying power. I have binding posts on the breadboard assembly so it is easy to keep the alligator clips apart from each other and I'm careful not to move the assembly while under power to avoid one of both of the alligator clips getting loose.

The antenna is optional. It's easy to add, just stick a wire into the breadboard. If you wish to use one, the length of the wire is fairly important. Make it a quarter wavelength long. To figure the length divide the speed of light, 300,000,000 meters per second by the frequency and divide this by 4. This gives you the length in meters. For my receiver, the calculation is (300,000,000/434,000,000)/4. This computes to 0.173 meters or 6.8 inches.

OK, so what do we do with this receiver setup? We take a 3.5mm stereo cable and plug it into the receiver circuit and into the line input of your PC or MAC. Then, you launch an audio recorder and editor program called Audacity. It's free, open source, software, and it's great. Here is a link to the site where you can download it. When the RF receiver receives the modulated signal from the Canon remote control transmitter it demodulates it (removes the 433MHz carrier) and outputs the intelligence it received from the remote control transmitter. This signal is within the audio range, so can be recorded by Audacity. Audacity, while capturing the signal displays the waveform like you would see on an oscilloscope. Once captured, you can zoom into any portion of the waveform. So let's look at some waveforms (Click on an image to enlarge):

Captured Data

The waveform above shows the receiver output zoomed all the way out. About 5 seconds after I started the recorder I pressed the exposure button on the camera remote control transmitter. What you see in the waveform is mostly a constant amplitude of random noise. Where it says "Pattern In Here" you can see a slight difference in the display. After a time, the noise reappears.

Exposure Pattern and Camera Preview Pattern. Both from Camera Remote Control Transmitter

Above shows two separate data captures where we have zoomed in to show about 50ms. of the captured data. This is within the "Pattern In Here" area. Each waveform shows one complete sequence of the camera's remote control transmit pattern. The bottom waveform, is the result of pushing the exposure button all the way down as if you were actually taking a photograph. The top waveform shows the pattern resulting in pressing the button partially down as if you were previewing the camera data or waiting for just the right moment to push the button all the way down to make the exposure. This preview mode is not particularly useful for my project. I just wanted to show that different patterns are possible. The difference in the patterns is slight. Look at the location at the black vertical line, and look at the very end of the pattern.

Pattern From Camera Remote Control Transmitter vrs. Pattern from My ATmega

In the above graphic, we see the a pattern captured when the camera remote control button is pressed (top waveform) compared with the waveform captured from my project's 433MHz transmitter driven by the ATmega microcontroller on my Gertboard. As you can see the two patterns are identical.

The two figures above were made possible because Audacity allows you to display multiple captures on the screen. You can then slide either capture left or right to line them up to see the differences or similarities between the waveforms. If you were a musician, each capture could be a track of your music. You can play one or more tracks while recording a new track. You could save the combination of multiple tracks as your song. Great stuff.

Now that we can see the pattern we have to analyze it. It was not particularity easy because, as you can see from the waveforms, the vertical edges were not very vertical. My method was to establish the time for the shortest pulse width. The beginning of the pattern has a low going pulse followed by a high going pulse, then low, high, and low going pulses. These five pulses are the shortest pulses. So everything is computed in relation to the time of these pulses. This pulse width is calculated to be 284 microseconds.

For clarity: Only the Beginning of the Pattern Is Shown

Rather than trying to break the pattern down to 1's and 0's, I resolved the pattern down to 6 sub-patterns. They are pretty well enumerated from the figure above. The only sub-pattern not shown is the long high going pulse at the end of the pattern. This "T" pattern is the length of 34 basic pulses. Therefore, the entire pattern can be expressed as:

Exposure: LHLHLEAABBAAAAAABABAABBABBBABBBBBBBABBLT

Preview:  LHLHLEAABBAAAAAABABAABBABBABBBBBBBABBBLT

Now that we have the patterns we can write some code. That's for the next posts.