Cyclic Redundancy Check (CRC) of DS18B20 Serial Data

This is a continuation of my last three blog posts.

Maxim has application note 27, "Understanding and Using Cyclic Redundancy Checks with Maxim 1-Wire and iButton Products", that I will refer to here. You can see this here.

If you transmit data from one place to another you want to know that the data arrived unchanged. The cyclic redundancy check is the best way to tell you if the transmission was successful.

We transmit serial data from two sources within the DS18B20: from its Read Only Memory which has the ROM code stored, and from the scratchpad memory. Basically, while transferring data to us, the user, the DS18B20 looks at the data it is sending, one bit at a time, and makes a calculation. The result of that calculation is an additional byte that is tacked on to the data stream. The value of this byte depends solely on the bits that have preceded it in the transmission. This additional byte is the CRC byte. The ROM code is actually 7 bytes long but the CRC byte makes it 8 bytes long. The scratchpad data is 8 bytes long and its CRS makes it 9 bytes long.

My code calculates the CRC as each bit of each byte is received. One of the nice things about CRC, and, this should be kept in mind, is if the transmission was successfully received, my calculation of the CRC will be 0. It's very easy to check for 0 in code. Any result besides 0 means the transmission failed.

How do you calculate CRC? It's hard to put that into words. The mechanism for calculating CRC is generally expressed as an electronic circuit. You can actually build this circuit and calculate the CRC, or you can emulate this circuit in software. I have done the latter in my One_wire_DS18B20 library file. I was surprised how few lines of code it took to write the CRC function. More on this later.

Getting Into Electronic Hardware

Here we go diving into interesting waters. But, I don't know how to otherwise discuss how this all works. So here we go. Maxim gives a representation of the electronic circuit in it's figure 2 of the application note. Here it is:

CRC Circuit from Maxim's Application Note 27

I have my own version of that figure that shows more detail, especially what is in those boxes called stages:

My CRC Equivalent Circuit

There are two different kinds of parts here, the boxes are called flip-flops (No connection to the ones you wear to the beach). Let's look at a flip-flop a little more closely:

Single D Flip-Flop

Why the funny name, flip-flop? The name describes the functionality, perfectly. The output from the part can be made to flip from one state to another. That is, from a high voltage to a low voltage; or, stated another way, from a logic HIGH state to a logic LOW state; or, stated yet another way, from a "1" to a "0". Of course going back in the other direction is just as easy. Flip-flops make up a large part of the circuitry in your computer microprocessor. There are thousands of them. The simplest can be constructed with just two transistors. The flip-flip we are showing here is a bit more complex and is called a D flip-flop. Below the flip-flop, is a timing diagram that plots voltage (vertical) against time (horizontal). When the voltage is at it's high state we call this a logic "1". When the voltage is near 0 volts, we call this a "0". (That is arbitrary, we could call a high voltage a "0" and a low voltage a "1").

We see three signals, or waveforms: "Data", "Clock", and "Output" (we are going to ignore reset). "Data" is the information that is being transmitted serially, one bit at a time. "Clock" is another signal, and when it makes a transition from a "0" to a "1", the logic level the device sees on its "D" pin, will appear at its "Q" pin. We show that the data on the "D" pin is "latched" on the rising edge of "Clock" by those upward arrows on the "Clock" waveform. The first time we see "Clock" going from a low to a high, the "Data" happens to be high (or a "1"). After a really short delay, that "1" appears at the output of the flip-flop. That little delay is important. It is not obvious why here, but will be when we consider more than one flip-flop. If you look at each rising transition you will see the output will mirror what is at the "D" pin. So far, not so interesting. But. wait, let's connect a second flip-flop to the first one.

Two Flip-Flops In Series

We connect a second D flip-flop so that its "D" ("D" for data, by the way) pin connects to "Q" of the first one. "Clock" connects to both flip-flops. Now we can see why that little delay is important. When we look at the second time the "Clock" goes from low to high, we see that the "D" pin of the first flip-flop is low, so the output of the first flip-flop wants to go low to follow the input. However, there is a slight delay so it stays high for a little while longer. If that delay was not there, the output of the first flip-flop would be changing at the exact time that "Clock" is latching the data at the input of the second flip-flop. What would the output of the second flip-flop be? A "1" or a "0"? Who knows! That little set-up delay assures that the input of the second flip-flop is a "1" long enough so the "Clock" latches a "1" at the output of the second flip-flop.

Is this circuit any more useful than the first? The usefulness of the circuit will be evident if we add more flip-flops. Let's have a total of 8:

Eight Bit Shift Register And Waveforms

Remember, "Data" is the information coming into the circuit one bit at a time. "Clock" captures that information in the first flit-flop. As new bits arrive, they cascade to the other flip-flops from left to right. Each rising edge of "Clock" shifts the information on "Data" from flip-flop to flip-flop. Thus, this circuit is called a shift register. The flip-flop outputs are labeled "Q7" - "Q0".

The circuit is also called a serial to parallel converter. After shifting in the serial data enough times to get the first bit cascaded down to the last flip-flop, the original eight bit data word is available on "Q7" - "Q0" in parallel format. We could connect "Q7" - "Q0" to other circuitry if we wished.

Why the grayed out areas? We don't have enough information to know if the logic levels in these areas are a "1" or a "0".

My project emulates the shift register in software. Every time we ask for data from the DS18B20, we call on a couple of functions in my 1_Wire library. We emulate "clock" with a "for" loop to shift in each bit of each byte. The "Q7" - "Q0" outputs are represented by an eight bit variable. We call that loop enough times to capture each byte in the message from the DS18B20.

As we shift in each bit, we do a calculation to determine the CRC value. We continue that calculation until the last bit of the last byte of the message has been shifted in from the DS18B20. For example, if we were capturing ROM code, we would capture eight bytes. The last byte of the ROM code is the CRC value (calculated by Maxim). After shifting in all 64 bits, the result of our calculation should be "00000000" (Recall my forth paragraph above).

Enough Background - Getting To The Cyclic Redundancy Check

Let's repeat that figure at the top of this post:

CRC Equivalent Circuit

Notice that this is similar to the shift register above. We have added three parts called exclusive OR gates. They each have two inputs on the left and one output on the right. The exclusive OR is one of a series of logic circuits. Some of the others are inverters (also called NOT gates), AND gates, and OR gates. These devices look at the logic levels at their inputs ("1" or "0") and decide what the logic level of their output will be. The output of the inverter is opposite of the input. The output of the AND gate is "0" unless all the inputs are "1". The output of the OR gate is a "1" if any of it's inputs are "1". The output of the exclusive OR gate will be "0" if the logic level of the inputs are the same (both "0" or both "1"). If the inputs are different ("0", "1" or "1", "0") the output will be a "1".

That CRC circuit seems to be a bastardized version of the shift register. The output from the last stage is fed back into the circuit. This circuit is specific to the way Maxim handles the CRC for devices that transfer data in eight bit bytes. They have a different circuit for devices that transfer 16 bit bytes. That circuit has 16 "stages"

The type of CRC circuit is also know by its polynomial, a mathematical expression representing the number of stages, and where, and, how many exclusive ORs are in the circuit. The polynomial for our circuit is shown on the figure above from Maxim's application. It is X⁸ + X⁵ + X⁴ + 1. This means we take the output of the forth, fifth, and eighth stage as inputs to our exclusive OR gates. There are other schemes with different number of exclusive OR gates that feedback to different stages. These have different polynomials.

The mathematics of the cycle redundancy check is quite involved and I won't begin to discuss it here. There is plenty of in-depth information on the web. Let's move on to the final discussion of implementing the CRC in software.

Calculating the CRC in Software and in Our Equivalent Circuit

The following code is taken from my calculateCRC_byte function found in my One-Wire_DS18B20 library.

The Equivalent CRC Circuit diagram contains the value of the CRC in the eight bits of "Q7" - "Q0". Note, however, the sketch variable "CRC" is an int, not a byte, making it 16 bits long. The reason "CRC" is 16 bits long is due to lines 11 and 14 of the sketch. On those lines, "CRC" is operated upon by a value that is nine bits long. Hence, "CRC" must be longer than 8 bits. When we reach the bottom of the function, the top eight bits of "CRC" will always be "00000000". The lower eight bits will be equivalent to "Q7" - "Q0" in the circuit. The top eight bits will also be "00000000" when we arrive at the "IF" statement in line 10.

The sketch has two loops, an outer loop for the number of bytes we send to the function, and an inner loop that runs eight times, once for each bit of the incoming byte, "inByte[i]".

Lines 9 Through 15 of the Code

Comparing the equivalent circuit to the sketch, the "IF" statement in line 10 implements, exactly, the exclusive OR gate on the left of the equivalent circuit. Let's look at that statement closely:

"if ((CRC % 2) ^ (inByte[i] % 2)){"

The "%" operator is called a modulo. The result of the modulo operation is the remainder after dividing whatever is to the left of the operator by whatever is to the right of the operator. In this case, we divide the current value of the variable "CRC" by 2. If the value of the variable "CRC" is even, the modulo will be "0". If odd, the module will be "1". Whether or not "CRC" is even or odd depends solely upon the value of its least significant bit (last bit on the right). Therefore, the value of "CRC % 2" is the value of the least significant bit of "CRC".

The CRC Equivalent Circuit holds the value of the CRC on its outputs "Q7" - "Q0". Applying the same argument as in the last paragraph, "CRC % 2" is the value of "Q0", the least significant bit of the CRC stored in the flip-flops.

Using what we learned above, "inByte[i] % 2", will give us the value of the least significant bit of the data byte, "inByte[i]". This is exactly the same as the current value of "Data" in the CRC Equivalent Circuit.

The symbol "^", in the "IF" statement, sitting between the two modulo expressions, is the bitwise exclusive OR operator. Because, "Q0" and "Data" connect to the exclusive OR at the left of the equivalent circuit, the entire "IF" statement is equivalent to that same exclusive OR gate. The result of the "IF" statement is precisely the same as the output of the left-hand exclusive OR gate. From our discussion of how the exclusive OR works, if the value of the least significant bit of the variable "CRC" and the value of the least significant bit of the array "inByte[i]" are the same (both "0" or both "1"), the result of the "IF" statement will be "0", otherwise (one is a "0" and the other a "1") the result will be a "1". In the same manner, if the value of "Q0" and the value of "Data" are the same, the output of the exclusive OR gate in the equivalent circuit will be "0". If different, the value will be "1". This output is the input to the first flip-flop at the left of the circuit, and one of the inputs of the other two exclusive OR gates in the equivalent circuit.

We described of the exclusive OR gate as being a "0" if its inputs were the same and a "1" if they were different. We used that description to evaluate the "IF" statement and the output of the exclusive OR at the left of the equivalent circuit. There is another useful way to look at an exclusive OR gate, and we will use that way in the following discussion. We can look at one input of an exclusive OR as a control input that controls the relationship between the second input and the output of the gate. If this control input is a "0", the output of the exclusive OR will follow the other input. If the control is a "1", the output will be opposite of the input (A "0" becomes a "1" and a "1" becomes a "0").

The result of the "IF" statement controls which of the two operations we apply to the value of "CRC" (either line 11 or line 14). We either exclusive OR "CRC" with "100011000" or "000000000". In our new way to look at the exclusive OR gate, those two series of nine bits are the control inputs to nine exclusive OR gates. They control what happens to value of the variable "CRC" at that point in the sketch. We can call them a control word.

Let's operate on a sample byte to see what those control bits do:

011010101	Sample	011010101
000000000	Control Word	100011000
011010101	Result	111001101

Note that when the control word is "000000000", there is no change to the sample, the result is the same as the sample.

I said the nine bit control word was like having nine exclusive ORs. But, six of those bits are always "0", both in line 11 and line 14. Consequently, we can replace those six exclusive ORs with direct connections, meaning there are only three exclusive ORs. This happens to be the same number of exclusive OR gates in our equivalent circuit. In fact, they are in the same relative positions. The most significant bit of our control word corresponds to the left-hand exclusive OR gate. The two bits of the control word that can change, are in exactly the same place as the other two exclusive OR gates in the circuit.

The purpose of the exclusive ORs are to modify the value of CRC existing at "Q7" - "Q0", or the value of the variable "CRC", before there is further processing. If the control word is "000000000", there will be no change in the value of "CRC". Since, the effect of exclusive ORing "CRC" with "000000000" is no effect at all, lines 13 - 15 of the sketch are superfluous. They do not appear in my actual code. I only added them here because it made explanations easier.

Let's look at the output of our exclusive ORs, be they in the sketch or the equivalent circuit. Exclusive ORing with "000000000" was the result of the output of the "IF" statement being "0". Let's see what happens to the equivalent circuit when the output of the left-hand exclusive OR is "0". First, the input to the left-hand flip-flop will be "0". The output of the left-hand exclusive OR also connects to an input of the other two exclusive ORs. Let's consider the output of the left-hand exclusive OR to be the control input to the other two flip-flops. If this control input is "0", the output of the two flip-flops ("Q4" and "Q3") will be the same as their other inputs, meaning there will be no change to the CRC.

When the output of the "IF" statement is "1" we will modify the value of the variable "CRC" with "100011000". At the equivalent circuit, the output of the left-hand exclusive OR will be a "1". The effect will be that the input of the left-hand flip-flop will be "1" and the control input to the other two exclusive ORs will be a "1". This will cause the "X4" to be of the opposite logic level of "Q4" and "X3" to be of the opposite logic level of "Q3". In other words, we have flipped those two bits.

Lines 16 and 17 of the Code

"Whew", we now have the variable "CRC", and the inputs of those flip-flops, set up for the next operation.

In the equivalent circuit, the signal labeled "Clock", which has been at "0" will briefly go to a "1" and back to "0". As in our discussion of the shift register, the logic level at the inputs of those flip-flops will now appear at the outputs of the flip-flops. This has the effect of shifting the current value of CRC down one position. Sometime after that, the value of "Data" will have the next bit of the serial data coming into the circuit.

Our sketch has an exact equivalent to the "clock" signal. That equivalent is line 16: "CRC >>= 1;". This operation will take the value of "CRC" and shift all 16 bits down one bit position to the right. A "0" will be shifted into the most significant bit of "CRC".

The next line, line 17, shifts the eight bits of "inByte[i]"down one bit to the right. A "0" will be shifted into the most significant bit. The operations of lines 16 and 17, are directly comparable to applying the clock to the flip-flops, and introducing the next data bit, in the equivalent circuit.

An Example

Let's take an example of two bytes and walk them through the sketch. Let's assume the two bytes are "10110100" and "01010011". If I run "10110100" through the sketch, by itself, the resulting CRC will be "0000000001010011". If we ignore the top 8 bits, the CRC is the same as the second byte we will pass to the sketch. Since the second byte is the same as the CRC of the first byte, the final CRC should be "0000000000000000", after processing both bytes.

First Byte 1:

i	j	inByte[i]	CRC	CRC Before Right Shift
0	0	10110100	0000000000000000	0000000000000000
0	1	01011010	0000000000000000	0000000000000000
0	2	00101101	0000000000000000	0000000100011000
0	3	00010110	0000000010001100	0000000010001100
0	4	00001011	0000000001000110	0000000101011110
0	5	00000101	0000000010101111	0000000010101111
0	6	00000010	0000000001010111	0000000101001111
0	7	00000001	0000000010100111	0000000010100111
0	7	00000000	0000000001010011	<-After the Final Shift

Now Byte 2:

i	j	inByte[i]	CRC	CRC Before Right Shift
1	0	01010011	0000000001010011	0000000001010011
1	1	00101001	0000000000101001	0000000000101001
1	2	00010100	0000000000010100	0000000000010100
1	3	00001010	0000000000001010	0000000000001010
1	4	00000101	0000000000000101	0000000000000101
1	5	00000010	0000000000000010	0000000000000010
1	6	00000001	0000000000000001	0000000000000001
1	7	00000000	0000000000000000	0000000000000000
1	7	00000000	0000000000000000	<-After the Final Shift

The DS18B20 transmits data least significant bit first. If you had an application that transmits most bit significant first you would have to modify the "IF" statement. Assuming the data coming in is 8 bits in length, you could replace "inByte[i] % 2", with "inByte[i] > 127". Also, "inByte[i] >>= 1" would be replaced with "inByte[i] <<= 1". If the CRC is calculated using a different polynomial, you would have to modify line 11 accordingly.

Feel free to download and experiment with this sketch. You can enter as many bytes as you wish. Just add some print statements and setup() and loop() to feed the function.

Adding Graphical Users Interface to Graphing Temperature Measurements

I started a series of blog posts back in July about making temperature measurements with sensors using the one-wire interface to the Pi. I described the hardware, software requirements, and the python script I wrote to make and report the measurements.

Earlier this month, I added graphing capability using RRDtool and PyRRD. The python script is, of course, included. Prior to that post, I presented a tutorial on programming with RRDtool and PyRRD. The graphical results are pretty impressive. My only complaint concerns actually running the script. Before making any measurements, the user has to answer quite a few questions that appear on the terminal. Answering all of those questions, each time I run the script, got a bit tedious, so I thought about adding a graphical users interface, i.e. popping a window for those questions and answers. This post reports on those efforts.

Before proceeding any further, I hope you readers have taken a look at my post proceeding this one (Oct. 18). While developing and testing my GUI I ran into a serious problem. The script must be run from root because of access to the the GPIO hardware on the Pi. I found that Linux would not give me permission to access the graphical system running as root. Look at that post to see how that problem was handled.

When you launch the script here is the default window that pops up. I'll talk about why I mentioned default window in a little bit:

Default Window

Taking this from the top of the window, I'll discuss each of the widgets, in turn: I have two temperature sensors, one on the breadboard the other on the end of a cable. With the radio buttons I can choose either sensor, or choose both sensors. You might think two check boxes would be more appropriate than the three radio buttons. But, the radio buttons work better with the program that actually makes the measurements (the program that calls for the GUI window).

Next are two entry boxes for applying a legend to the graph for each sensor. The default has only the breadboard sensor enabled because of the radio buttons selection above, which is why the cable entry box is grayed out. Select cable sensor and the entry box for cable can be edited and the breadboard entry box is grayed out. Select both sensors and neither entry box is grayed out. If you don't type anything in the entry boxes the default text (as shown) will appear on the graph. RRDtool has its own rules and some may seem strange. Here, for example, if you wish to have a colon in your legend (cable: water temperature), you must escape the colon with the backslash character (cable\: water temperature).

After the legend comes the title for the graph. A default string is included. RRDtool does not require you to escape colons in the title, but does require you to escape spaces.

Comments are optional. You must escape colons here too, just like the legends.

Next we can choose the graph background color. Black, I think, looks good on the screen, but if you wish to send the graph to your printer, it would use an awful lot of ink. Consequently, we have an option of having a graph with a white (actually light gray) background, You can choose to create both graphs if you wish. In the future, I would like add the possibility of a custom color graph. That would have new windows popping up from the basic window, something I want to try coding. Like the situation with the sensor widgets, check boxes might seem to be more efficient, bur radio buttons work better with the script making the measurements.

The width of the graph is hard-coded to 600 pixels, but the height is programmable with the slider control. You can choose values from 100 to 400 pixels in increments of 100. A shorter graph is useful when the variation in the measurement values is small (nearly a straight, horizontal line). I have seen where RRDtool actually repeats values on the Y axis if the graph height is large and there is a small variation in measurement values. RRDtool selects the values on the Y axis, not the programmer.

Next, we have the number of measurements and the interval between measurements. The smallest interval being one minute.

Finally, we come to the matter of file names. As of now (meaning I may alter this in the future), the directory name is hard-coded in the script. I have a directory for all temperature measurements, but make up a new sub-directory, under that, based on the date (for example: 2013_10_25 for Oct. 25, 2013). The file name asked for by the widget is a base name without extension. Depending on the number of sensors used, and the number of graphs generated, we will make three to five files for each run of the script. The measurements used for the graph are stored in files with an .rrd extension - one file for each sensor. There is a .png file for each graph (one for black background, one for white). Finally, there is a .txt file generated to store all of the results, along with the measurement times, in an easily read form (the .rrd files are not easily read). This .txt file is another feature that is new with this version. If, for example, the base file name is rodger and we were to use both sensors, and generate both graphs, we would generate the following five files: rodger.txt, rodger_bread.rrd, rodger_cable.rrd, rodger_black.prn, and rodger_white.prn.

The check box to the left of the file name is to protect the files from being overwritten if they already exist. It looks in the directory with the current date. Putting a check mark in the box allows the files to be overwritten.

Let's put some values into the entry boxes but we'll make errors in all of the boxes:

Result Of Pressing Continue If There Are Entry Errors

After pressing the "Continue" button, the red error messages show up to the right of the entry boxes. The legend and comment boxes have non-escaped colons and the title has a non-escaped space. The measurement interval can not be zero (the number of measurements will give the same error if they total to zero). The error message will appear if a non-numeric character appears in the number of measurements or any of the measurement interval boxes.

There are three possible error messages that can appear to the right of "Base Filename". They are:

• Only numbers, letters, and underscore
• Will overwrite existing file ("Allow File Overwrite" not checked
• Must enter a file name (if you leave the box empty)

Once all of the corrections have been made, hitting the "Continue" button will kill the window and allow the program that makes and graph temperature measurements to proceed. If you hit the "Quit" button the window closes and the calling script terminates.

One other action happens before the window closes. All of the parameters entered in the window widgets are saved into a configuration file. The next time the program is run, all of the parameters are loaded into the window rather than the default values. This should save a lot of time if the same run is repeated or only a few changes are made. If a lot of changes are to be made a press of the "Default" button will bring up all of the default values.

Let's do an actual run. The cable sensor is placed into a 12oz. glass of hot tap water and allowed to cool to ambient temperature. Here is the window:

Window For Actual Run

After pressing the "Continue" button the window closes and you see the terminal window:

Terminal Window After a Few Measurements

The terminal window shows the parameters chosen on the GUI. It gives you a last look at your selections. If you decide you wanted something else, simply hit Ctrl C to terminate the script. A careful reader will notice the parameters on the terminal screen do not match the parameters chosen in the window above. That is because I forgot to save the screen shot, so two different runs are represented.

Below the parameters it says that the 1-wire modules had to be loaded. They are not loaded upon boot-up, so every time the Pi is powered up, or rebooted, the 1-wire modules must be loaded. The script checks to see if the modules are loaded, and, if not, loads them. Subsequent runs of the script will not need to load the modules if power stays applied and the Pi is not rebooted. Next, the script displays when the first measurements will be made. Recall that measurements are synchronized with the measurement interval. Since one minute was selected, we wait for the seconds to be 00. Next, we see the display of several measurements showing the day of the week, date and time, sensor, and the temperature as measured. The measurement results, along with time and sensor, are also displayed on my 16 character by two line Led Display.

After all of the measurements have been made, as stipulated by the value in "Number Of Measurements", the script will stop. If you wish to stop the script prematurely, you can simply issue a Ctrl C or press the switch on the breadboard.

Terminal Window At The End Of The Run

Once the script is terminated, the terminal display looks similar to the figure above. This information is from yet another run. I did not coordinate that aspect of this post very well. You see the last of the measurements followed by a salutation and the reporting of the fact that there were no glitches.

Originally, I had problems with measurement failures where the device file could not be read. I call that a glitch and keep track of the number of glitches for the entire run. My solution to recover from these glitches is to unload and reload the 1-wire modules. For every glitch, I only unload and reload the modules a maximum of three times. If there is no recovery after the tree tries, I stop the script and report a message. This problem has not reoccurred, I have not seen any glitches for a long time.

The start time and stop time, are followed by long numbers. These numbers are the number of seconds since January 1, 1970. This is how RDTool records numbers in the .rrd files. If you want the time on the graph to be correct, you must apply these long numbers to the measurements. I report those numbers at the end here in case you wish to investigate the .rrd files. The information in the .rrd files look like gibberish if viewed with a text editor. You have to issue a RRDtool command to see the contents of these files. For more information see my post "RRDtool For Dummies Like Me" under the topic "What Measurement Values Go Into the Database".

Let's look at the code. The code is divided into two scripts, one for the GUI and the other for the main program that makes, graphs, and records the temperature measurements. The reason for a separate script for the GUI was to minimize changes to the main script. The main script is derived from the code shown in my blog entry, "Graphing Real Temperature Data Using RRDtool and PyRRD". The code for the GUI was developed independently, and has test code at the bottom so it can be run by itself, to check its operation. The GUI script was made a callable module by saving it as a .pyc file. The first 425 lines of the GUI code becomes a single function, guiwindow(). All of the parameters collected in the window are passed to the main script by line 103 of the main script:

     variable_list = guiwindow()

Code For the GUI Window:

I know it's a rather long script (another reason for making it separate from the main script), but a lot goes on here. Each element of the window (widgets) must be defined. Some of the widgets, when clicked on by the user, spurn actions. These actions must be defined. A lot of code is devoted to making the appearance correct. Placement of widgets is somewhat of a challenge. As in most GUI applications, user inputs are checked to make sure required parameters are not missed, or errors made. The operator is made aware of these errors by messages so he knows what to correct. The error checking requires a lot of code. Just look at the function proceed(). Most of the widgets have error messages included in their definations.

The development of the GUI is done using a module called Tkinter. It is probably available for all, common, Linux distributions for the Pi. I have developed all of my code using Python 2.7 so my line to import Tkinter is: "from Tkinter import *". In Python 3.x, that line would be "from tkinter import *". Tkinter is not just for Python, and not just for Linux.

So, how does one get started developing GUIs for their Raspberry Pi projects? There are many references, including "The Python Library Reference". This document points you to other sources, including the Python Wiki, a source with even more sources. One source you need to have available, constantly, is "Tkinter 8.5 Reference: a GUI for Python" from the Computer Center of New Mexico Tech. You need this for no other reason then getting the syntax correct. It is really a great asset to GUI development. Another good source of help is my code above, and code of others, for you to see practical examples. There are even YouTube videos with Tkinter tutorials. There is a ton of stuff out there on the net on Tkinter.

There are a few other items of interest in this script. Note the use of the Subprocess module in the function "getdirectory()". This is a way of issuing the Linux Ls command from Python. Here, Ls is used to see if a directory has been previously made. If it does not exist, the function makes the directory. Another cool thing is what happens, after the user presses "Continue" and the inputs have been found to contain no errors. Before the window closes and operation passed to the main program, all of the user's input gets saved to a file. When the window is opened again, the information in this file populates the widgets. This is done using a module called Pickle (actually cPickle. A version developed using C that runs faster than Pickle). Check it out, Pickle is a very efficient way to do the job and saves you from writing many lines of code.

Main Script For Making, Graphing, and Recording Temperature Measurements:

Since this code for the main script was discussed in my earlier post, I'm not going to say much about it. One element I added is saving the results to a text file. I discussed this earlier in the post. Lines 281 - 288, 312 - 319, and 327 - 334 handle this task. The directory and file name come from the GUI window.

I almost forgot, here are the two graphs:

Graph Corresponding To Window Above

This Graph Is Better For Printing - It Saves Ink

Graphing Real Temperature Data Using RRDtool and PyRRD

My previous blog post, "RRDtool for Dummies Like Me" was a tutorial on the use of RRDtool and PyRRD to graph data from a python script. For simplicity, I used made-up data. This post talks about plotting real data taken with the Raspberry Pi.

In June and July of this year, I wrote five blog posts about measuring temperature with the Dallas Semiconductor DS18B20 One-wire temperature sensor. The temperature measurements were reported to the terminal screen and my 16x2 LCD display (seen in photo in blog header) so would not be saved. I did modify the script to send the results to a file but have not reported on that effort. Perhaps that would make a worthwhile post for later.

It just seemed natural to want to plot that data to see the change of temperature with time. That led to my investigation of RRDtool and PyRRD, and to the tutorial in my last post. This post combines temperature measurements with plotting in a python script.

I have two DS18uB20 temperature probes in my system. One looks like a small transistor and sits on the breadboard and the other is at the end of a cable. For the graph below, I put the cable sensor in a glass of hot tap water while the breadboard sensor measured ambient air temperature.

Plot With Real Temperature Data

The python code follows.

The script is an evolution of code I reported on previously. All the One-wire temperature code was discussed in my June and July posts. The plotting code grew out of the code in the last post. The code for the LCD display came from my posts of March and April of this year.

My python script may seem pretty large, but I wanted to make it useful, and to incorporate all of the hardware connected to the Pi that makes sense to use. These are some of the features:

1. Choose to use either sensor, breadboard or cable, or both sensors.
2. Choose graph characteristics:
   1. If desired, give the breadboard, and/or cable sensor another name.
   2. Give the graph a title - mandatory
   3. If desired, enter a comment that prints on the graph.
   4. Select graph with either black or white background, or choose to make both graphs. White background is best to print. It's easier on printer ink. I think black looks better on the screen, like as seen in the sample graph.
   5. Enter the height from 100 to 400 pixels. If you are plotting from one sensor of a temperature that does not change much it is better to choose a lower value for the height. Otherwise, RRDtool may repeat the values on the Y axis, like: 70, 70, 70, 71, 71, 71, 72, 72, 72, etc.
   6. Choose a file name. Up to four files will be created with that name: an .rrd file for each sensor chosen, and a .png file for each background color chosen.
3. Choose the maximum number of measurements to make for each sensor. The script will stop once that number of measurements (for each sensor) is reached. In the future, I may modify the script so that once the maximum is reached, new measurements replace the oldest measurements. Remember, this is how a round robin database works.
4. Choose the time interval between measurements, in minutes. For each sensor: The first measurement will be taken at the first time corresponding to the time interval. For example, if the time is now 37 minutes past the hour, and 15 minutes is selected as the time interval, the first measurement will be at 45 minutes past the hour. If you select the time interval to be one day, the first measurement will be made at midnight. When making the plot, RRDtool will only place a measurement point at a time corresponding to the nearest measurement interval. Again, as an illustration, if we choose a day as the time interval, and we made a measurement at 8PM, the plot will show the measurement at midnight, and --- unless it is the first measurement, the value will be different. That value is not even the linear interpolation of the measured value. See my past post. Strange, but that is the way the tools work.
5. After entering all of the user information, you have a chance to review and enter the information again, if you are not satisfied.
6. You can choose not to proceed after entering the user information. This will stop the script.
7. As well as plotting the temperature data, the average temperature for for each sensor, is calculated and displayed below the plot.
8. The current measured temperature value, for each sensor, is displayed on the 16 character by 2 line LCD display.
9. All measurements are sent to the terminal along with the time of the measurement.
10. There are three ways to stop the script:
    1. When we have reached the maximum number of measurements.
    2. Pressing CTRL-C from the keyboard (Issuing a keyboard interrupt).
    3. Hitting the switch on the breadboard.

There were two things that did not work out as I wished. The first is that RRDtool/PyRRD makes adding some text to the graph somewhat difficult. The title requires you to escape all spaces and probably some punctuation. A space needs to be "\ " in the variable (title_it). It's so easy to forget to add that backslash that I tried to have the script do that for you. It would seem that the following would work:

      title_it = title_it.replace(" ", "\ ")

It doesn't. It replaces a space with "\\ ". This is a title prints on the graph: This\ is\ a\ title, which looks pretty dumb. The following does not work either, giving the same result:

      title_it = title_it.replace(" ", r"\ ")

Since I had no luck adding the backslashes before the spaces, I have the script, in the function check_title, check that the user added the backslashes. Note that the user type "\ " for each space, but the function looks for "\\" before each space (line 103). That makes sense, the first backslash escapes the second backslash.

Here's the second thing that did now work out: I wanted to handle all of that user input with dialog boxes. Using stdio is a bit clunky and if you decide to change some input, you have to enter everything again. Besides, I wanted to learn to incorporate those features. Python has a nice module called Tkinter which brings the graphic users interface to python and other programming languages.

I experimented with a simple program to bring up a dialog box to enter or choose a file name. It worked just fine. I could run it from a Idle Python shell, or from the stdio. No problem. Then I incorporated some GPIO code which meant I had to run the script as root. Now it failed. It threw an exception saying, "Client is not authorized to connect to the Server .....". By client, I assume it means root. Obviously, if the user is me, it works fine, but not if the user is root.

A Pi In The Fridge

I did find a practical use for the the temperature measurements using the Pi (see my past several blog posts). We thought the temperature in our refrigerator was a bit too high. To check it out, I put the Pi with the DS18B20 temperature sensors in the refrigerator. The Pi was powered by SparkFun's solar charger and battery back. The Pi was on a lower shelf while the cable sensor was in a glass of water on an upper shelf. The whole assembly was in the refrigerator for several hours. It all worked well at the temperature that got down to just under 40F (a little too warm for the food though).

Raspberry Pi on Lower Shelf of Refrigerator

The lower left corner of the switch (white square with the 4 printed on it) is pointing to the DS18B20 To-92 sensor. It's just a small black blob in the photo (not one of my best photographs. I didn't want to hold the refrigerator door open too long to get that perfect shot). The three pin connector for the DS18B20 cable sensor is just under the TO-92 sensor. I made that little connector to interface the three leads of the cable to the breadboard.

Probe In Glass of Water - Hunk of Cheese Holding Cable in Place

A Raspberry Pi Thermometer - My Software To Display Temperature

Let's back up for a moment. Two posts ago, I indicated that the Raspbian and Occidentals distributions have support for the 1-Wire interface and the DS18B20 temperature probes. We see this implemented in the kernel modules w1-gpio and w1-therm. I said that the only support I knew of was to read the temperature, and that you could not write to the device to change the resolution or set temperature limits. There is one other limitation I forgot to mention: You can only use GPIO 4 to interface the device to the Pi - no other choice is possible.

I wrote a python script using my two temperature sensors. It takes repeated measurements but is not greatly practical as it does not store the measurements. I'm going to add that capability next. I guess I'll start by writing the results to a file. I'm going to investigate RRDTool which is a way to put the results into a database. The program includes graphing capabilities, which sounds fascinating.

I kind of overloaded my script with a lot of stuff, because, after all, I'm doing this to learn. I don't really have a pressing need to make temperature measurements. I do this for fun. Luckily, I don't have to program to please a client, or employer, or to comply with any deadline. The Pi, along with python programming, is just an adult toy.

Functionally, the script asks the user how frequently to make measurements and which of the two probes to use. The program outputs each measurement to the terminal screen along with the date and time and the sensor choice. It also outputs the temperature, along with the date and time, to my 16 character by 2 line LCD display. The user terminates the script by issuing a keyboard interrupt (CTRL-C) or by pressing the button on the breadboard. The script then tells you when you started the run and the duration of the run. Here is a sample from a very short run:

Screen Shot of Terminal Output After a Short Run of the Script

Readout on the LCD Display

Without further ado, here is the script:

I would like to talk about what I find interesting about the script. I'll take it from the top and work down.

Lines 17 - 23: We need to import quite a few modules. I'll come back and talk about the commented out module in line 19 later. Line 23 is the module I wrote that has the functions to control the 16X2 LCD display. You can see this code on my blog entry "16 X 2 LCD Base Code" of March 23, 2013.

Lines 25 - 36: Here's a bit of extravagance. The only lines here that are necessary are lines 34 and 35. All the other lines in the function are superfluous. This function looks to see if the two modules w1-gpio and w1-therm are loaded in the kernel. If not, it loads them. When you load the modules, run the script. and then terminate the script, the modules stay loaded in the kernel. If you run the script again, it's OK if the script says to load them, if they are already loaded. My checking to see if they are loaded is unnecessary. But, I'm not doing this for a living, so a few extra lines is no big deal.

When I saw lines 34 and 35 in every other python program dealing with the DS18B20, I was very curious about that Linux command "modprobe". What did it do? I found this site.It has everything I needed to know: how to load modules, remove modules, and see what modules are present in the kernel. I wanted to use this information in my script, thus all those extra lines.

The loadmodules function demonstrates two methods to run a Linux command in Python. One uses the Popen method in the subprocess module (line 29), and the other uses the system method in the os module (lines 34 and 35). The second method is useful for commands that do not return anything you are interested in knowing about. It would be the one to use if you wanted to run "cd" to change directory. If you wanted to list the contents of the directory using "ls", the Popen would be the one to use. To see if the w1-gpio and wi-therm modules are loaded we issue the "lsmod" command, which lists the loaded modules like "ls" does for directories and files. The output is redirected to the stdio using the "stdio = subprocess.PIPE" optional argument. In the same way, we redirect any error to the stdio.

There appears to be an error in either line 31 or lines 34/35. But they are correct. If the two modules are loaded, the output of "lsmod" shows the module names with the underscore. When we load the modules, we use the dash in the names. Strange.

Lines 38 - 67: This function is rather long because it captures the occurrences of the w1-slave file failing to open. I talked about this in previous posts. If that glitch never occured, only six lines would be necessary (38, 47, 48, 63, 64, and 65). We use the Popen method to read the two line w1-slave file in line 47 (see the last post to see examples of these two lines). The output is redirected to the stdio and we read the contents in line 48. We do not have to bother to close the file. Lines 47 and 48 are a good alternative to using the open command, followed by the read command, and the close command. Note in line 48, we also read any error, such as ".....file not found" and can easily deal with the error. Using the open command, if the file is not found, then an exception is thrown and we have to handle that exception. Not a big deal, but the Popen approach seems much more efficient.

If an error occurs, my approach to recover is to unload the modules using the modprobe -r command and then reload them after a short delay. I wait 10 seconds at the end to give plenty of time for the modules to load. I tried to recover from the error by simply rereading the file (reissuing the Popen command), but that did not work. Once the error occurred, it kept occurring until the script was terminated. Unloading and reloading the modules does seem to work. I only do the unload and reload module process three times. If there is still an error, the script stops, by the statement raise(IOError). We'll talk about this statement when we discuss the main program. For my own information, I keep track of the occurrences of w1-slave failing to open in the variable glitches.

Lines 69 - 83: The print2display function simply takes the current time and the temperature and outputs the former to line 1 and the latter to line two of the LCD display. I found that those 10ms. delays were required to keep the display from going berserk after a time.

One of the fun aspects of this project was playing with the methods in the datetime module. The current time is determined by the now method in the datetime.datetime module and passed to the print2display function. The strftime method is a way to format the time in a variety of ways. Look here at section 8.17. in the Python documentation.

Lines 85 - 104: This is another example of throwing something in for the fun of it. Before the program terminates I wanted to print the duration of the run. Why do this? Simply because I wanted to figure out how to do it. I had read about the method timedelta in the datetime module in the Python documentation but could not understand it. That was because I thought timedelta took two date/times and reported the difference - just what I wanted. I was wrong, timedelta applies a time difference to a date/time, and reports a new date/time. I did make use of timedelta in the commented out line 92. More on this later.

At the beginning of the main part of the program I save the current time in the variable starttime. To use starttime in my function, I declared it a global variable. I could have easily passed the value of starttime to the function totaltime. I have a feeling that would have been the preferred method. To find the elapsed time, I simply subtract the starttime from the time at the end of the run and make a string of the result.

Now is a good time to discuss the commented out line 92 and the commented out line 19. I left those lines in the script for the purpose of discussing testing. You should test as much of your scripts as possible to assure they will work under various circumstances, and, for this script, will work with long duration runs. The string in line 97, the value of elapsedtime can look very different if the run time is over or under a day. Let's say the run time was 23 hours, 59 minutes, and 59.99 seconds. The string value of elapsedtime will be '23:59:59.990000'. If the run duration is one minute longer, elapsedtime looks like this: '1 day, 0:00:59.990000'. Quite a bit different. For testing this, I didn't want to make a run of over one day, so that's is where I used line 92. Using timedelta, I just added 24 hours to elapsedtime to get the string with the 1 day in it. Worked like a charm. You can see how I parsed the string value of elapsedtime to print the time in days, if needed, then hours, minutes, and seconds.

Lines 106 - 116: There is a momentary switch on the breadboard (that square job with the number 4 printed on it). It's purpose, here, is to provide a second way to exit the script. Checking the switch position is a matter of looking at the GPIO pin connected to the switch. If it is a logic high, the switch is not pressed (pin is pulled up by a resistor to 3.3V). Pressing the switch grounds the pin. If the function sees the pin low, it waits 200ms. and looks again. If the pin is still a logic low, it assumes the switch is really being pressed. The final action is the statement raise(KeyboardInterrupt). I'll reserve comment about this until later.

The Main Program:

Lines 131 - 137: One of the elements I really wanted to incorporate into my script was error handling, and error creating. These seven lines are a small manifestation of error handling which is implemented with the try and except statements. When a script encounters an error that is not "handled" within the script, it just stops and you receive an error message. The try and except statements are what you use to "handle" error messages. If an error occurs while processing the try block (the indented code below the try: statement), execution moves from the try block to the except block. Once the code in the except block is executed, execution moves to the code beyond the except block. For example, when asked to input a value to measurement_interval you input a letter rather than a number. Without the error handling, python will stop the script because measurement_interval expects a float, not a string literal. Python throws an exception, and prints: "ValueError: could not convert string to float" Since we don't want the program to stop because of a simple keystroke error, we trap the error by executing the except block. That block has the one word command of "pass", which means do nothing. The except block must have something in it, so "pass" suffices. So, if you press a letter instead of a number, absolutely nothing happens and you go back to the beginning of the while loop. Once you enter any number, including zero, you exit the loop with the break command with a good value for measurement_interval.

Line 125: These are the sub-directories representing my two temperature sensors. The first one is the breadboard sensor while the second one is on the end of the cable. Since this is all a learning experience, and I was reading about dictionaries, I was going to make this line: device_id = {breadboard: "28-00000400d39d", cable: "28-000004986cbb"}. But, I didn't get around to it. Tuples are pretty efficient though.

Lines 152 - 187: Here is the main loop of the script. I want to discuss the error handling first. The entire while loop is a try block so if any error occurs while executing within the loop, including execution within one of the functions, execution will jump out of the try block (thus out of the while loop) to one of the except blocks following the try block. The first two except statements look for specific errors. The first except statement looks for a keyboard interrupt, as you get by typing CTRL-C. This is one of the two normal ways I have provided to end the script. If, indeed CTRL-C was pressed, the code within that first except block (line 178) is processed and execution passes to line 189, bypassing the next two except blocks.

When executing the read_temp_raw function, if after three attempts we fail to open the w1-slave file, we get to the statement: raise(IOError) (line 67). The raise statement actually creates an error, in this case, IOError. If that indeed occurs, we jump out of the read_temp_raw function, out of the while loop in the main program, and land in the except(IOError) block. After executing the two print statements, we jump to line 189.

If some other, unanticipated, error occurs, execution passes out of the try block, bypasses the first and second except blocks, and lands in the third except block. Any error except a keyboard interrupt, or an IOError will be processed by that bare except statement. The print statement says we have an unexpected error and prints out what that error is. Once this is done, we move to line 189.

Within the while loop

Now let's look within the while loop. If the read_temp_raw function can open w1-slave, it returns the two line contents of w1-slave as a two element list. We check to see if the first line ends in 'YES'. If it does not, which seems to be the case about half the time, we wait a short time (the value of the variable short_wait) and go back to the beginning of the while loop to try again with a new read of w1-slave. If we do find a 'YES' we get the contents of the end of the second line following 't='. If 't=' is not found (rare if ever), we do like we do if 'YES' is not found: simply wait a short time and return to the top of the while loop and try again. If we do find 't=' we read the five digit number following it. That number is divided by 1000 to get the temperature in Centigrade and then converted to Fahrenheit.

After the temperature is found the current time is determined. The temperature and time along with the probe in use is printed to the screen and sent to the print2display function. It was rather a fun endeavor to come up with that print statement. I talked about the string formatting method, strftime, previously. Finding the unicode value ((hex B0) for the degree character was a challange. Finally, the time to wait between measurements is set to the value of measurement_interval inputted by the user.

Previously, I had the code to check the switch in the while loop in the main part of the program. Also, the wait between measurements was implemented, simply, with a sleep statement with the value of measurement_interval. This meant that if measurement_interval was a long time, in minutes or hours, I would have to press the switch at exactly the right time to catch the code outside of the sleep time, or press the switch for a long, long time. With the code in lines 170 - 174, I check the switch every second so the longest time to press the switch before the script stops is one second (actually 1.2 seconds to account for the wait statement in the check_switch function). It is necessary to check the switch in line 170 before getting into the small while loop in case the user selected zero seconds for measurement_interval.

I have just discovered a glitch in this script. If the user selects a negative number or a non-whole number for measurement_interval, that while loop will never get to exactly 0. I'll have to fix that. I should have tested for that before

Finally, how does the switch provide an exit from the script? It does so by the raise(KeyboardInterrupt) statement in line 116. If the switch is pressed, the effect is the same as if the user typed CTRL-C.

A Raspberry Pi Thermometer - The Hardware

This is my first post in quite a while. I have, however, been busy with my Pi - just havn't been writing about it. There will be upcoming posts about my Gertboard and my camera remote control/motion detector project. The project uses the ATmega microcontroller on the Gertboard, so I got to use the Arduino IDE (Integrated Development Environment) and program in C rather than Python. I'll have a lot to say about the Gertboard, and not all I'll say is flattering. This post, however, is about a project that does not use the Gertboard, just the Pi and the breadboard that you see in the header of this blog, with the addition of temperature sensors.

I purchased two temperature sensors from Adafruit, both are DS18B20 1 wire devices. The only difference between them is one looks like a TO-92 package transistor and the other is packaged into metal, waterproof, cylindrical case (6 mm diameter, 30 mm long) with a 36" integrated, shielded, cable. Adafruit also sells a higher temperature version. Actually, what makes it a higher temperature device is that it has a PTFC cable that allows it to to be used up to the 125°C rating of the sensor. The sensor is the same. The less expensive version, I purchased, has a PVC cable. Adafruit suggests not to expose the PVC cable to over 100°C.

Both versions (we'll consider the two waterproof versions as the same), are the same electronically, and will measure between -55°C to 125°C (-67°F to +257°F). They measure with 9 to 12 bit resolution. The more bits, the longer it takes between measurements. The default is 12 bit resolution and it can take a measurement about every 750ms. The remarkable thing is they only have three leads (the waterproof model has the cable shield brought out along with the three leads). One lead is ground, another is power which can be from 3.0Vdc to 5.5Vdc. The third pin is for data, used to write to and read from the device. The data line is "open collector" (when not driven to a logic low, acts like it is not connected to the circuit), therefore it is necessary to include a "pull-up" resistor between the data pin and the source of power that the device connects to (in the case of RaspberryPi, 3.3V). The open collector configuration allows you to connect the data pin of more than one device to the same Pi GPIO pin (of course, only one device can talk to the Pi at any one time). Adafruit, ships the devices with a 4.7K resistor. In fact, it is possible to ignore the power pin and power the device from the data pin (via the pull-up resistor. This mode complicates the timing and thus the programming, so I don't think I will try that feature.

My project is installed on my breadboard that has a 16 character by two line LCD display and a push button switch installed. Please see my previous blogs where I use these devices. My projects with the temperature sensors, detailed in subsequent blogs include:

1.    Making a temperature reading every two seconds and reporting the readings, along with the date and time, to the terminal

2.    Doing the same thing but also displaying temperature, date, and time to the LCD display

3.    Doing all the same but making a reading every minute

4.    Doing all the same but logging the results to a file.