Serial Transmission - Gertboard (Like an Arduino) to Raspberry Pi

Introduction

If you read my post of March 14, you know I have an enclosure with temperature sensors, and a Raspberry Pi with a Gertboard attached that connects to a receiver board. The enclosure transmits sensor data to the receiver board. The receiver passes the decoded data to the ATmega328P microcontroller on the Gertboard. I covered the transmission, via 434 MHz transmitter and receiver in my March 14 post. Check it out here.

The last piece of the data chain is to pass the data to the Raspberry Pi. A python program, running on the Raspberry Pi, is in charge of obtaining the measurements from the C++ program running on the Gertboard's ATmega328P. As long as the sensor enclosure is transmitting, the Gertboard receives the measurements.

If you wonder why I pass data to the Raspberry Pi it is because I can get graphs like this:

I can get a log file like this (just the beginning of the file):

I can setup the measurement run with this graphical user interface I created:

Data Captured By the Receiver

ROM codes, and 12 bytes of description for each DS18B20 sensor, are stored in the EEPROM of the enclosure's ATMega328P. One temperature sensor is permanently connected to the enclosure, but I can connect as many other sensors as I wish. However, the ROM codes and descriptions for those other sensors must be stored in the EEPROM. When I apply power to the enclosure, we find out what sensors are attached by attempting to read from the scratchpad of each sensor for which we have a ROM code. If the CRC passes, we know the sensor is connected.

I have a full suite of ATmega328P, C++ sketches, for adding, editing, and removing sensor data from the EEPROM. Check it out here. I also wrote my own library for the DS18B20 temperature sensor, and other libraries. Check out my libraries here.

Let's assume I have data for five sensors stored in the EEPROM and sensor's 1, 3, and 5 are connected to the enclosure. I command each sensor, in turn, to make a temperature measurement. When the measurement is available, we assemble 20 bytes of data (data for one sensor). The first byte is always the number of sensors that are connected to the enclosure. For our example, that number is 3. The next to the last byte is the number of the sensor which corresponds to its position in the EEPROM. In our example, that will be 1, 3, or 5. The other bytes correspond to the temperature measurement, high and low alarm temperature, and the sensor description. The last byte is the CRC. We continually make measurements running through each of the connected sensors.

The receiver, continually, looks for valid data. If it recognizes synchronization pulses (see my March 14 post), it sees that the data is valid. It decodes the 20 bytes of Manchester encoded data for the sensor that happened to be transmitting. It may be sensor 1, 3, or 5. It looks at the first byte of data to find the number of sensors in the measurement run. In this case it will be three. An array is initialized whose length is the number of sensors, three in our case, times the number of data bytes per measurement, which is 20. In our example, the array will be 60 bytes long. The data for the first sensor is placed in the array. Next, the receiver code looks for, and captures, data for the next two measurements. This data is appended to the array. Now we have 60 bytes of data. The order of the data may be for sensors 1, 3, and 5; or 3, 5, and 1; or 5, 1, and 3.

After accumulating the 60 bytes, the receiver code looks to see if the Raspberry Pi's python program is requesting the data. If not, the data in the array is overwritten with the next series of measurements. There will be a minimum of one minute between requests from the Pi to the receiver (selected by the user). Depending upon the resolution of the sensor and how power is applied to the sensor (normal or parasitic) the receiver will have those 60 bytes available about every 300ms. to every 2200ms. Therefore, you can see that most of the measurements are lost.

Receiver Code

I have taken another fragment of the C++ code running on the Gertboard's ATmega328P. Visit here to see the entire script.

The code fragment includes the function, "transmit_data()" that looks for a request from the Pi and transfers the data to the Pi. I also included some of the code in "loop()" to illustrate the sequence of events.

To initiate a data transfer cycle, we first look for synchronizing pulses by running line 41 (see blog post of March 14 for this function). We stay here until we find valid synchronization. Once synchronization is confirmed, we move to line 42, the "manchester_data()" function. This will return data for one sensor - the sensor that was transmitting at the time. The code for the "manchester_data()" function can also be found at my March 14 post. If the CRC passes, we now read the first byte of the data to see how many sensors are connected to the enclosure. Now, that we know how many sensors, we can declare the array "xmit_data[]". This array will hold all of the data for all of the sensors for one measurement cycle. After populating "xmit_data[]" with the data from the one sensor we have acquired, we repeat "synchronize()" and "manchester_data()" for each of the other sensors in the run. Now, we have all of the data. Next is to see if the Raspberry Pi has called for the data.

We are now down at line 68 assuming there were no CRC errors. Line 68 takes us to the function "transmit_data()". The function looks to see if there is anything on the ATmega's serial bus (line 15). If so, it is read and put into the array "buff[]" by implementing "Serial.readBytes()". We are looking for one character, and that character is "s". If we find the "s", we place all the data, one byte at a time, onto the serial bus (via the print statement in line 19). If there was no request from the Pi for data, the contents of "xmit_data[]" will be overwritten with new data.

Python Code Running In the Raspberry Pi

The following is a fragment from a rather long program running on the Pi. The program handles setting up the log files for storing the data, sets up the graph parameters, obtains sensor information (description, high and low alarm settings, and resolution), calls the GUI program and incorporates the user responses, makes and records the temperatures, and creates the graphs. The entire program is here. This program calls another python program that handles the graphical user interface. That program is here.

The function "number_of_sensors()" is called once to obtain the sensor information: description, alarms, and resolution. Then, it is called every time the program calls for temperature measurements. When the function exists, if there was valid data, all of the data from all of the sensors will be in the global array "recv_data[]". In addition, the function returns the number of sensors in the run.

"number_of_sensors()" calls the function "retrieve_serial()". "retrieve_serial()" does the actual work of requesting data from the ATmega328P on the Gertboard, and retrieving the response from the Gertboard. When requested by "number_of_sensors()", it clears the serial port of lingering data, writes the character "s" to the serial port, and waits for a response from the Gertboard. It will check every second for data. If it sees no data, or if the length of the data does not equal a multiple of 20 bytes, it keeps looking. If it has not found valid data after 10 seconds, it gives up and returns "0". If valid data is seen, it returns the number of bytes received.

Manchester Encoding/Decoding Data Between Devices

My most recent posts have been about making temperature measurements with the Maxim/Dallas DS18B20 temperature sensors. I have an enclosure with a 2 line by 16 character display that can connect to many of these sensors.

If I want to go beyond merely displaying the temperature, and wish to record and graph the temperature, I wirelessly transmit the data from the enclosure to my Raspberry Pi. Consequently, I need some sort of transmitter and receiver. The transmitter and receiver I chose operate at 434MHz and are small and simple. Both are available from Sparkfun Electronics.

On the left: Temperature Sensor Enclosure. On the right, 454 MHz Receiver connecting to the Gertboard. The Gertboard plugs into the RaspberryPi (seen at the very top).

Sparkfun's 454 MHz Transmitter on the left. Their 454 MHz Receiver on the right.

I don't transmit directly to the Pi because I want the receiving device to be dedicated to processing the received data, A microcontroller is ideal for that type of task. A multitasking microprocessor, like the Pi, cannot do this very well - it doesn't focus on one task. The Pi, however, is really good at logging and storing data. My Pi has a marvelous capability to graph temperatures (See my posts about RRDTool).

My receiver connects to my Gertboard, which, in turn, connects to my Pi. The Gertboard has the same microcontroller device as an Arduino Uno: an Atmel ATmega328P. My temperature measuring enclosure also has the same microcontroller.

My next post will tell how my data gets from the Gertboard to the Raspberry Pi. It's an interesting story and deserves a post of its own.

Asynchronous Data Transfer

I am sending data asynchronously, it can start at any time. The receiver must be able to find the start of a burst of data. For that to happen, I precede the data with a synchronization pattern. This pattern is chosen so that it can never be confused with the actual data. I accomplish this by encoding the actual data in a way that there are never more than two logic 1's or two logic 0's together. My synchronization pattern, then, must have more than two 1's and/or two 0's, together.

I use Manchester encoding with my data, which is a standard protocol. Every logic 1 in the data is replaced with a 01, while a logic 0 becomes a 10. You can see how this avoids more than two 0's or 1's together. For example: 0000 becomes 10101010. Only where there is a change in logic level will you find two 0's or 1's together. For example: 001100 becomes 101001011010.

There is a good reason for avoiding long strings of 1's or 0's: the receiver will see this an an interruption of data. The receiver has an automatic gain control, and if it sees no data, it will increase its gain to the maximum. The receiver will then output rail to rail thermal noise (See the beginning of the yellow scope trace below).

The disadvantage of this scheme is that I must transmit two bits for every single bit of data.

Here is what the start of my data looks like on the scope:

Red trace: Data to the Transmitter. Yellow trace: Received Data

The upper trace (red) is the data to the transmitter, while the lower trace (yellow) is the received data at the Gertboard.

After every burst of transmitted data, there is a lapse of several hundred milliseconds (transmitter outputs 0V). During this time one if the temperature sensors makes its measurement and converts the measurement to two bytes of digital data. At some point during this dead time, the receiver's automatic gain control goes to its maximum. You can see the resulting noise in the beginning of the trace of the receiver's output.

You can clearly see the two synchronization pulses in both the transmitted and received data. The synchronization pulses consist of four high bits followed by four low bits. The synchronization pulses are preceded by a 101010 pattern. The receiver will recognize this pattern as actual data and will adjust its automatic gain control. There is 10 pattern following the synchronization pulses. The start of the Manchester encoded data follows that.

My Data

I am sending 20 bytes of data with each measurement:

• byte 0: Number of sensors being accessed
• byte 1 and 2: From the DS18B20 Scrathpad - the measured temperature
• byte 3: From the DS18B20 Scrathpad - Upper temperature alarm. Previously set by the user
• byte 4: From the DS18B20 Scrathpad - Lower temperature alarm. Previously set by the user
• byte 5: From the DS18B20 Scrathpad - Resolution (9, 10, 11, or 12 bits). Previously set by the user
• byte 6 through 17: From the ATmeta328P EEPROM - Sensor Description. Previously set by the user
• byte 18: The number of the sensor as determined by its position in the ATmega328P's EEPROM.
• byte 19: Cyclic Redundancy Check (CRC) byte calculated from the previous 19 bytes.

Except for byte 0, each burst of data contains the data from one temperature sensor. Each sensor is accessed, in turn, before starting with the first sensor, again.

Transmit Code

The following is a fragment of the code I usually run on my Temperature Sensor Enclosure. I say usually because I can upload other code to the box via its attached USB/FTDI port. The entire code can be seen here:

The enclosure has one permanently mounted DS18B20 temperature sensor and three receptacles for plugging in more sensors. It is possible to connect hundreds of these sensors to the enclosure (normal and parasitic power is supported). However, I only have room in the ATmega's EEPROM for the ROM codes and descriptions of 50 sensors. In the photo above, you can see I have two DS18B20 cables attached to the box. The code auto-detects the sensors when the code starts. As long as the ROM code and description of the sensor is in the EEPROM, the code will include the sensor in the measurements and in the transmitted data. If I have a new sensor, I have code to upload to the enclosure that will detect new sensors and will prompt for description, and alarm and resolution settings.

Code Fragment of Sketch Running On ATmega328P in Enclosure To Transmit Data to Gertboard.

Whenever we are ready to transmit data, the function "synchronize()" is called, followed by 20 calls to function "send_data()" - one call for each data byte in the burst. The timing is established by the global variable "bit_time". It is calculated to send data at 600 baud (1200 bps - remember it takes 2 Manchester encoded bits to send one bit of data).

Note my use of ATmega register programming rather than "bitMode" and "digitalWrite". I have a whole tutorial about register programming at by blog here:

Receive Code

The following is a fragment of code that runs on my Gertboard that receives Manchester encoded data from my Temperature Sensor Enclosure. The entire code can be found here.

Code Fragment of Sketch Running On ATmega328P on the Gertboard To Receive Data from the Temperature Sensor Enclosure.

While the transmit code is pretty easy to figure out, the receive code is interesting and bears some analysis. First, it illustrates the use of ATmega interrupts. Not shown in the code fragment are two lines in setup():

EICRA = B00000001; // Any change will trigger interrupt

EIMSK = B00000001; // Enable INT0 interrupt

This means that any change in logic level of the ATmeta328P pin connected to the receiver circuitry will trigger an interrupt. Put another way: if the logic level changes from 0 (0V) to 1 (3.3V) or from 1 (3.3V) to 0 (0V), an interrupt is issued.

Lines 18 - 20 constitutes the Interrupt Service Routine. Whenever an interrupt is issued, the variable "found_transistion" becomes true. We set "found_transistion" false, in lines 34, 41, 66, and 75. When the "while" statements in line 35, 43, 67, and 77 are reached, processing is halted until the interrupt occurs. Using the "time" and "duration" variables and the "micro()" function, the time between interrupts (corresponds to the time between transitions) can be determined.

Analyzing The Receive Data

Synchronization and Made-up User Data

When sychronize() is called from the main program, it looks for a positive pulse of 4 bits in duration (plus or minus 10%). It keeps looking until it find it. If it next finds a negative pulse of 4 bits in duration (plus or minus 10%), it exits the routine. It will stay in this function until it is satisfied it has found the synchronization pattern.

Once we drop out of the sychronize() function, the manchester_data() function is called. We stay in that function until 20 bytes are received. Looking at the waveform above, we start in the function at point "c". Next we look for one more transistion (lines 66 and 67) to bring us to point "d".

We look for the actual received data by entering the outer "do" loop at line 72, performing the inner "do" loop (line 74) 20 times. The logic level of each bit of data we find is tracked by the variable "present_bit". This starts as 0 because the 10 pattern following the synchronization pulses is equivalent to a Manchester code of 0.

We are at point "d" looking for the next transition which occurs at point "e". Using the threshold of 1.5 times the time of one bit (in global variable bit_time), we determine if we have encountered a change in logic level from the original 0 to a 1. Since the time from point "d" to "e" is obviously less than the threshold, we have not encountered a change in logic level. Therefore, the first bit received is a 0. The "if/else" statements in lines 80 to 86 determine the actions taken if we do exceed, or do not exceed the threshold. Since we have not, "present_bit" does not change and the variable "no_clocks" is incremented by 1.

The variable "no_clocks" has two functions. First, it indicates when we have received a bit of data. Since we must receive two bits of Manchester encoded data for each bit of sensor data, we take no action if "no_clocks" is an odd number ("if" statements lines 87 to 90). If "no_clocks" is even, we capture the last bit by shifting the contents of variable "hold_byte" to the left, and inserting the value of "present_bit" at the least significant position of "hold_byte".

The second function of "no_clocks" is to tell us when we have received an entire byte of sensor data. This will be the case if "no_clocks" reaches 16. If that is the case, the value of "hold_byte" is transferred to the next byte of the array "frame[]". When we have received all 20 bytes of "frame[]" we exit the function.

When last we were looking at the waveform, we were at point "e". We return to the beginning of the inner "do" loop to look for the next transition (point "f"). The time from "e" to "f" is also less than the threshold. We return back to the beginning of the inner do loop and find the next transition (point "f" to "g"). This time we find the threshold is exceeded so the value of "present_bit" flips to a 1 and "no_clocks" increments by 2. I think the action is easy to follow from here.

DS18B20 12 Bit Resolution - What does that mean

DS18B20 Resolution

The following is from figure 2 of the Maxim Integrated data sheet: "DS18B20 Programmable Resolution 1-Wire Digital Thermometer"

Temperature Register Format

When the data sheet talks about 12 bit resolution, what do they mean? From the figure above, you can see that two 8 bit bytes are reserved for the temperature measurement. That means there are a total of 16 bits. But, notice there are only 12 unique bits. Bits 11 to 15, of the most significant byte (MS Byte), will always be the same. If one bit is a "1", all 5 bits will be a "1". If one bit is a "0", all 5 bits will be a "0". That is where the 12 comes from: 12 unique bits.

How is that number related to resolution? Of those 12 bits, eight are reserved for the integer part of the number: bit 4 to bit 11. Bit 0, bit 1, bit 2, and bit 3 make up the fractional component. You can choose to have the DS18B20 measure with 9, 10, 11, or 12 bit resolution. If you choose 12 bits, all 12 bits are used in the calculation of temperature. If you choose 11 bits, the least significant bit will always be a 0. If you choose 10 bits, the last two bits will always be zero. If you choose nine bits, the last three bits will always be zero giving you only one fractional bit of resolution.

With nine bits, the least significant bit is bit 3. The value of bit 3 (from the figure above) is 2^-1. That is the equivalent to ¹⁄₂ or 0.5. So, nine bits gives you a resolution of 0.5°C.. If you choose 12 bit resolution, the least significant bit has a value of 2^-4. That gives you a resolution of ¹⁄₁₆ or 0.0625. So, 12 bits gives you a resolution of 0.0625°C..

Why would you want to choose anything other than 12 bit resolution? Why not always choose the best? The answer is speed. When asked for a temperature conversion, the DS18B20 takes multiple measurements and averages them before reporting the result. The higher the resolution, the larger the number of measurements go into the average, and the longer time to complete the conversion. The maximum conversion time for 12 bit resolution is eight times the maximum time for nine bit resolution. Here is a chart of resolution in bits, degrees, and maximum conversion time:

Number of Bits	Resolution	Maximum Conversion Time, ms.
9	0.5000°C	93.75
10	0.2500°C	187.5
11	0.1250°C	375
12	0.0625°C	750

Deriving Temperature From the Register Data - Temperatures below 0°C

The following is Figure 1 from the Maxim data sheet:

Temperature/Data Relationship

I hope it is clear how the decimal value of temperature relates to the binary digital output column when the temperatures are in the positive range. It might not be as clear when the temperatures are negative. Those binary numbers are really signed binary numbers. This means that the most significant bit, bit 11, or in this case, bits 11 through bit 15 (since these five bits are always the same), tell you if the number is positive or negative. If those bits are all "0"'s the number is positive. If those bits are all "1"'s the number is negative. However, that is only half the story. You might think that if +25.0625 is 0000 0001 1001 0001 then -25.0625 could be 1111 1001 1001 0001. But you see from the figure above that is not the case.

As small as the DS18B20 is, it has to have some computing power. Therefore, it shares characteristics with your microprocessor or microcontroller device. These devices all have some electronic circuitry that perform basic functions. One such function is to add binary numbers. Another is to change all the "1"'s to "0"'s and all the "0"'s to "1"'s. That operation is called inverting. A function that is not commonly implemented is circuitry to perform subtraction. That is because it is possible to subtract by using the two circuits I described, adding and inverting.

How would you subtract 13 from 25 without actually performing subtraction. Let's assume we were working with 8 bit signed binary numbers. 25 is 0001 1001. 13 is 0000 1101. Subtracting 13 from 25 is the same as adding -13 to +25. In order to facilitate the subtraction process, devices like the DS18B20 (because they must do some computing) represent negative numbers by what is called the two's complement of the number. The two's complement is formed by using the aforementioned processes, adding and inverting. First we invert then we add 1.

Let's get the two's complement of -13. First, we invert the 0000 1101 to get 1111 0010. Next, we add 1 yielding 1111 0011. When we add that to 0001 1001 (25) we get 1 0000 1100. Since we are using 8 bit signed binary numbers, that lonely "1" on the left gets lost giving us our final result of 0000 1100 which is decimal 12. This is how the DS18B20 represents the binary (and hex) temperatures that are below 0°C: by their two's complement. Let's prove it. If +25.0625 is 0000 0001 1001 0001. What is -25,0625 in two's complement? Let's invert 0000 0001 1001 0001 to 1111 1110 0110 1110 and add 1. This gives us 1111 1110 0110 1111, which is what is shown in the figure above.

If you want more information on two's complement, including some theory, I recommend this site.

Calculating Temperature From DS18B20 Register Values

When you get the results from the DS18B20, two bytes from the scratchpad, how do you convert them to temperature? Without getting theoretical, this is what I do:

1. Multiply the value of the most significant byte (scratchpad byte 1) of the temperature data by 256.
2. Add to the result of step 1, the the value of the least significant byte (scratchpad byte 0).
3. Check to see if the temperature is negative by testing if the result from step 2 is over 127. If so the temperature is negative.
4. If the temperature is negative, subtract the result of step 2 from 65536.
5. Divide by 16