Serial Transmission - Gertboard (Like an Arduino) to Raspberry Pi

Introduction

If you read my post of March 14, you know I have an enclosure with temperature sensors, and a Raspberry Pi with a Gertboard attached that connects to a receiver board. The enclosure transmits sensor data to the receiver board. The receiver passes the decoded data to the ATmega328P microcontroller on the Gertboard. I covered the transmission, via 434 MHz transmitter and receiver in my March 14 post. Check it out here.

The last piece of the data chain is to pass the data to the Raspberry Pi. A python program, running on the Raspberry Pi, is in charge of obtaining the measurements from the C++ program running on the Gertboard's ATmega328P. As long as the sensor enclosure is transmitting, the Gertboard receives the measurements.

If you wonder why I pass data to the Raspberry Pi it is because I can get graphs like this:

I can get a log file like this (just the beginning of the file):

I can setup the measurement run with this graphical user interface I created:

Data Captured By the Receiver

ROM codes, and 12 bytes of description for each DS18B20 sensor, are stored in the EEPROM of the enclosure's ATMega328P. One temperature sensor is permanently connected to the enclosure, but I can connect as many other sensors as I wish. However, the ROM codes and descriptions for those other sensors must be stored in the EEPROM. When I apply power to the enclosure, we find out what sensors are attached by attempting to read from the scratchpad of each sensor for which we have a ROM code. If the CRC passes, we know the sensor is connected.

I have a full suite of ATmega328P, C++ sketches, for adding, editing, and removing sensor data from the EEPROM. Check it out here. I also wrote my own library for the DS18B20 temperature sensor, and other libraries. Check out my libraries here.

Let's assume I have data for five sensors stored in the EEPROM and sensor's 1, 3, and 5 are connected to the enclosure. I command each sensor, in turn, to make a temperature measurement. When the measurement is available, we assemble 20 bytes of data (data for one sensor). The first byte is always the number of sensors that are connected to the enclosure. For our example, that number is 3. The next to the last byte is the number of the sensor which corresponds to its position in the EEPROM. In our example, that will be 1, 3, or 5. The other bytes correspond to the temperature measurement, high and low alarm temperature, and the sensor description. The last byte is the CRC. We continually make measurements running through each of the connected sensors.

The receiver, continually, looks for valid data. If it recognizes synchronization pulses (see my March 14 post), it sees that the data is valid. It decodes the 20 bytes of Manchester encoded data for the sensor that happened to be transmitting. It may be sensor 1, 3, or 5. It looks at the first byte of data to find the number of sensors in the measurement run. In this case it will be three. An array is initialized whose length is the number of sensors, three in our case, times the number of data bytes per measurement, which is 20. In our example, the array will be 60 bytes long. The data for the first sensor is placed in the array. Next, the receiver code looks for, and captures, data for the next two measurements. This data is appended to the array. Now we have 60 bytes of data. The order of the data may be for sensors 1, 3, and 5; or 3, 5, and 1; or 5, 1, and 3.

After accumulating the 60 bytes, the receiver code looks to see if the Raspberry Pi's python program is requesting the data. If not, the data in the array is overwritten with the next series of measurements. There will be a minimum of one minute between requests from the Pi to the receiver (selected by the user). Depending upon the resolution of the sensor and how power is applied to the sensor (normal or parasitic) the receiver will have those 60 bytes available about every 300ms. to every 2200ms. Therefore, you can see that most of the measurements are lost.

Receiver Code

I have taken another fragment of the C++ code running on the Gertboard's ATmega328P. Visit here to see the entire script.

The code fragment includes the function, "transmit_data()" that looks for a request from the Pi and transfers the data to the Pi. I also included some of the code in "loop()" to illustrate the sequence of events.

To initiate a data transfer cycle, we first look for synchronizing pulses by running line 41 (see blog post of March 14 for this function). We stay here until we find valid synchronization. Once synchronization is confirmed, we move to line 42, the "manchester_data()" function. This will return data for one sensor - the sensor that was transmitting at the time. The code for the "manchester_data()" function can also be found at my March 14 post. If the CRC passes, we now read the first byte of the data to see how many sensors are connected to the enclosure. Now, that we know how many sensors, we can declare the array "xmit_data[]". This array will hold all of the data for all of the sensors for one measurement cycle. After populating "xmit_data[]" with the data from the one sensor we have acquired, we repeat "synchronize()" and "manchester_data()" for each of the other sensors in the run. Now, we have all of the data. Next is to see if the Raspberry Pi has called for the data.

We are now down at line 68 assuming there were no CRC errors. Line 68 takes us to the function "transmit_data()". The function looks to see if there is anything on the ATmega's serial bus (line 15). If so, it is read and put into the array "buff[]" by implementing "Serial.readBytes()". We are looking for one character, and that character is "s". If we find the "s", we place all the data, one byte at a time, onto the serial bus (via the print statement in line 19). If there was no request from the Pi for data, the contents of "xmit_data[]" will be overwritten with new data.

Python Code Running In the Raspberry Pi

The following is a fragment from a rather long program running on the Pi. The program handles setting up the log files for storing the data, sets up the graph parameters, obtains sensor information (description, high and low alarm settings, and resolution), calls the GUI program and incorporates the user responses, makes and records the temperatures, and creates the graphs. The entire program is here. This program calls another python program that handles the graphical user interface. That program is here.

The function "number_of_sensors()" is called once to obtain the sensor information: description, alarms, and resolution. Then, it is called every time the program calls for temperature measurements. When the function exists, if there was valid data, all of the data from all of the sensors will be in the global array "recv_data[]". In addition, the function returns the number of sensors in the run.

"number_of_sensors()" calls the function "retrieve_serial()". "retrieve_serial()" does the actual work of requesting data from the ATmega328P on the Gertboard, and retrieving the response from the Gertboard. When requested by "number_of_sensors()", it clears the serial port of lingering data, writes the character "s" to the serial port, and waits for a response from the Gertboard. It will check every second for data. If it sees no data, or if the length of the data does not equal a multiple of 20 bytes, it keeps looking. If it has not found valid data after 10 seconds, it gives up and returns "0". If valid data is seen, it returns the number of bytes received.

Printing Leading Characters With Arduino IDE Serial Monitor

I have a blog post on calculating and checking the Cyclic Redundancy Check (CRC) byte when transmitting serial data. Here is a link. Near the bottom, is a sketch fragment of my CRC function and a table. The table illustrates how the CRC is calculated as the function progresses, bit, by bit, through the data. Here is the output I received from the Arduino IDE Serial Monitor:

Arduino IDE Serial Monitor Output Printing Binary Numbers

That's really ugly. The serial monitor that is part of the Arduino IDE does not print leading zeros. I wanted to see those zeros. "CRC" is an integer, so has 16 bits. "inByte[i]" is 8 bits long. I wrote a small function to calculate how many leading zeros to print when printing a binary number. Then, I expanded it to to handle numbers of any base, and finally, to add any leading character, not only zeros. This is the output using my leading character function:

Serial Monitor Output Using My Leading Character Function

My leading character function follows. It is amazingly simple. However, it's not universal. It does not handle minus signs or decimal points:

The "pow" operator raises the value in "base" to the power of "position" - 1. For example, if you wish to print a 16 bit binary number, "factor" will be 2¹⁵. You can see that "factor" can become a very large number, especially if you are dealing with decimal or hex numbers. Consequently, "factor" is a double, a floating point data type. However, the function may not work if "factor" wants to be too large. Just imagine if you wanted 16 characters of a decimal number. "factor" will be 1,000,000,000,000,000. A double data type for all Arduinos beside the Due is four bytes in length, not enough bytes to express 10¹⁵. The Due may be able to handle 10¹⁵ - it's double is 8 bytes in length.

Leading spaces can come in handy for printing dates. You can use it to line up dates like this:

• April   9, 2015
• April 10, 2015

Manchester Encoding/Decoding Data Between Devices

My most recent posts have been about making temperature measurements with the Maxim/Dallas DS18B20 temperature sensors. I have an enclosure with a 2 line by 16 character display that can connect to many of these sensors.

If I want to go beyond merely displaying the temperature, and wish to record and graph the temperature, I wirelessly transmit the data from the enclosure to my Raspberry Pi. Consequently, I need some sort of transmitter and receiver. The transmitter and receiver I chose operate at 434MHz and are small and simple. Both are available from Sparkfun Electronics.

On the left: Temperature Sensor Enclosure. On the right, 454 MHz Receiver connecting to the Gertboard. The Gertboard plugs into the RaspberryPi (seen at the very top).

Sparkfun's 454 MHz Transmitter on the left. Their 454 MHz Receiver on the right.

I don't transmit directly to the Pi because I want the receiving device to be dedicated to processing the received data, A microcontroller is ideal for that type of task. A multitasking microprocessor, like the Pi, cannot do this very well - it doesn't focus on one task. The Pi, however, is really good at logging and storing data. My Pi has a marvelous capability to graph temperatures (See my posts about RRDTool).

My receiver connects to my Gertboard, which, in turn, connects to my Pi. The Gertboard has the same microcontroller device as an Arduino Uno: an Atmel ATmega328P. My temperature measuring enclosure also has the same microcontroller.

My next post will tell how my data gets from the Gertboard to the Raspberry Pi. It's an interesting story and deserves a post of its own.

Asynchronous Data Transfer

I am sending data asynchronously, it can start at any time. The receiver must be able to find the start of a burst of data. For that to happen, I precede the data with a synchronization pattern. This pattern is chosen so that it can never be confused with the actual data. I accomplish this by encoding the actual data in a way that there are never more than two logic 1's or two logic 0's together. My synchronization pattern, then, must have more than two 1's and/or two 0's, together.

I use Manchester encoding with my data, which is a standard protocol. Every logic 1 in the data is replaced with a 01, while a logic 0 becomes a 10. You can see how this avoids more than two 0's or 1's together. For example: 0000 becomes 10101010. Only where there is a change in logic level will you find two 0's or 1's together. For example: 001100 becomes 101001011010.

There is a good reason for avoiding long strings of 1's or 0's: the receiver will see this an an interruption of data. The receiver has an automatic gain control, and if it sees no data, it will increase its gain to the maximum. The receiver will then output rail to rail thermal noise (See the beginning of the yellow scope trace below).

The disadvantage of this scheme is that I must transmit two bits for every single bit of data.

Here is what the start of my data looks like on the scope:

Red trace: Data to the Transmitter. Yellow trace: Received Data

The upper trace (red) is the data to the transmitter, while the lower trace (yellow) is the received data at the Gertboard.

After every burst of transmitted data, there is a lapse of several hundred milliseconds (transmitter outputs 0V). During this time one if the temperature sensors makes its measurement and converts the measurement to two bytes of digital data. At some point during this dead time, the receiver's automatic gain control goes to its maximum. You can see the resulting noise in the beginning of the trace of the receiver's output.

You can clearly see the two synchronization pulses in both the transmitted and received data. The synchronization pulses consist of four high bits followed by four low bits. The synchronization pulses are preceded by a 101010 pattern. The receiver will recognize this pattern as actual data and will adjust its automatic gain control. There is 10 pattern following the synchronization pulses. The start of the Manchester encoded data follows that.

My Data

I am sending 20 bytes of data with each measurement:

• byte 0: Number of sensors being accessed
• byte 1 and 2: From the DS18B20 Scrathpad - the measured temperature
• byte 3: From the DS18B20 Scrathpad - Upper temperature alarm. Previously set by the user
• byte 4: From the DS18B20 Scrathpad - Lower temperature alarm. Previously set by the user
• byte 5: From the DS18B20 Scrathpad - Resolution (9, 10, 11, or 12 bits). Previously set by the user
• byte 6 through 17: From the ATmeta328P EEPROM - Sensor Description. Previously set by the user
• byte 18: The number of the sensor as determined by its position in the ATmega328P's EEPROM.
• byte 19: Cyclic Redundancy Check (CRC) byte calculated from the previous 19 bytes.

Except for byte 0, each burst of data contains the data from one temperature sensor. Each sensor is accessed, in turn, before starting with the first sensor, again.

Transmit Code

The following is a fragment of the code I usually run on my Temperature Sensor Enclosure. I say usually because I can upload other code to the box via its attached USB/FTDI port. The entire code can be seen here:

The enclosure has one permanently mounted DS18B20 temperature sensor and three receptacles for plugging in more sensors. It is possible to connect hundreds of these sensors to the enclosure (normal and parasitic power is supported). However, I only have room in the ATmega's EEPROM for the ROM codes and descriptions of 50 sensors. In the photo above, you can see I have two DS18B20 cables attached to the box. The code auto-detects the sensors when the code starts. As long as the ROM code and description of the sensor is in the EEPROM, the code will include the sensor in the measurements and in the transmitted data. If I have a new sensor, I have code to upload to the enclosure that will detect new sensors and will prompt for description, and alarm and resolution settings.

Code Fragment of Sketch Running On ATmega328P in Enclosure To Transmit Data to Gertboard.

Whenever we are ready to transmit data, the function "synchronize()" is called, followed by 20 calls to function "send_data()" - one call for each data byte in the burst. The timing is established by the global variable "bit_time". It is calculated to send data at 600 baud (1200 bps - remember it takes 2 Manchester encoded bits to send one bit of data).

Note my use of ATmega register programming rather than "bitMode" and "digitalWrite". I have a whole tutorial about register programming at by blog here:

Receive Code

The following is a fragment of code that runs on my Gertboard that receives Manchester encoded data from my Temperature Sensor Enclosure. The entire code can be found here.

Code Fragment of Sketch Running On ATmega328P on the Gertboard To Receive Data from the Temperature Sensor Enclosure.

While the transmit code is pretty easy to figure out, the receive code is interesting and bears some analysis. First, it illustrates the use of ATmega interrupts. Not shown in the code fragment are two lines in setup():

EICRA = B00000001; // Any change will trigger interrupt

EIMSK = B00000001; // Enable INT0 interrupt

This means that any change in logic level of the ATmeta328P pin connected to the receiver circuitry will trigger an interrupt. Put another way: if the logic level changes from 0 (0V) to 1 (3.3V) or from 1 (3.3V) to 0 (0V), an interrupt is issued.

Lines 18 - 20 constitutes the Interrupt Service Routine. Whenever an interrupt is issued, the variable "found_transistion" becomes true. We set "found_transistion" false, in lines 34, 41, 66, and 75. When the "while" statements in line 35, 43, 67, and 77 are reached, processing is halted until the interrupt occurs. Using the "time" and "duration" variables and the "micro()" function, the time between interrupts (corresponds to the time between transitions) can be determined.

Analyzing The Receive Data

Synchronization and Made-up User Data

When sychronize() is called from the main program, it looks for a positive pulse of 4 bits in duration (plus or minus 10%). It keeps looking until it find it. If it next finds a negative pulse of 4 bits in duration (plus or minus 10%), it exits the routine. It will stay in this function until it is satisfied it has found the synchronization pattern.

Once we drop out of the sychronize() function, the manchester_data() function is called. We stay in that function until 20 bytes are received. Looking at the waveform above, we start in the function at point "c". Next we look for one more transistion (lines 66 and 67) to bring us to point "d".

We look for the actual received data by entering the outer "do" loop at line 72, performing the inner "do" loop (line 74) 20 times. The logic level of each bit of data we find is tracked by the variable "present_bit". This starts as 0 because the 10 pattern following the synchronization pulses is equivalent to a Manchester code of 0.

We are at point "d" looking for the next transition which occurs at point "e". Using the threshold of 1.5 times the time of one bit (in global variable bit_time), we determine if we have encountered a change in logic level from the original 0 to a 1. Since the time from point "d" to "e" is obviously less than the threshold, we have not encountered a change in logic level. Therefore, the first bit received is a 0. The "if/else" statements in lines 80 to 86 determine the actions taken if we do exceed, or do not exceed the threshold. Since we have not, "present_bit" does not change and the variable "no_clocks" is incremented by 1.

The variable "no_clocks" has two functions. First, it indicates when we have received a bit of data. Since we must receive two bits of Manchester encoded data for each bit of sensor data, we take no action if "no_clocks" is an odd number ("if" statements lines 87 to 90). If "no_clocks" is even, we capture the last bit by shifting the contents of variable "hold_byte" to the left, and inserting the value of "present_bit" at the least significant position of "hold_byte".

The second function of "no_clocks" is to tell us when we have received an entire byte of sensor data. This will be the case if "no_clocks" reaches 16. If that is the case, the value of "hold_byte" is transferred to the next byte of the array "frame[]". When we have received all 20 bytes of "frame[]" we exit the function.

When last we were looking at the waveform, we were at point "e". We return to the beginning of the inner "do" loop to look for the next transition (point "f"). The time from "e" to "f" is also less than the threshold. We return back to the beginning of the inner do loop and find the next transition (point "f" to "g"). This time we find the threshold is exceeded so the value of "present_bit" flips to a 1 and "no_clocks" increments by 2. I think the action is easy to follow from here.

Get Rid of MaAfee On Your Brand New Computer

I just purchased a new Windows laptop. Between all the bloat and the pre-installation of McAfee I was tempted to reformat the hard disk, purchase a copy of Windows 8.1, and start from scratch.

What a racket McAfee has. They give you a free subscription for a couple of months. So what do you do when that expires? I would bet the vast majority of people either do without virus protection, or pay McAfee every year to maintain their protection. Nearly everyone knows the dangers of being unprotected, so I would guess most simply pay McAfee.

I want the free Microsoft Windows Defender as my virus protection. With McAfee present, Defender is turned off. With a little web search, I found it was quite easy to remove McAfee and turn on Defender.

Everything you need to know to remove McAfee comes from this McAfee document.

I added one step to the process. I turned off my network connection.

This all applies to a new computer with Windows 8.1. It turned out to be really simple and painless. Here is what I did:

1. Downloaded the McAfee Consumer Product Removal Tool (MCPR) - main paragraph 2a. from the link I gave you.
2. Removed computer from my network because next step removes your protection.
3. Uninstall McAfee - Main paragraph 1. from the link. No problems here.
4. Find where you stored the MCPR.exe and run it. Just answer the questions asked. The prompt in paragraph 2f. never appeared so I did not have to deal with it.
5. When you see "CleanUp Successful" reboot your computer.
6. When you are up and running again connect yourself to your network.

Check the action center (white flag in the notification area of the taskbar). Click on "Open Action Center" and then click on "Security". You will see that your firewall is protected by Windows and your virus and spyware protection is now provided by Windows Defender. You don't have to do anything to turn it on.