Serial Transmission - Gertboard (Like an Arduino) to Raspberry Pi

Introduction

If you read my post of March 14, you know I have an enclosure with temperature sensors, and a Raspberry Pi with a Gertboard attached that connects to a receiver board. The enclosure transmits sensor data to the receiver board. The receiver passes the decoded data to the ATmega328P microcontroller on the Gertboard. I covered the transmission, via 434 MHz transmitter and receiver in my March 14 post. Check it out here.

The last piece of the data chain is to pass the data to the Raspberry Pi. A python program, running on the Raspberry Pi, is in charge of obtaining the measurements from the C++ program running on the Gertboard's ATmega328P. As long as the sensor enclosure is transmitting, the Gertboard receives the measurements.

If you wonder why I pass data to the Raspberry Pi it is because I can get graphs like this:

I can get a log file like this (just the beginning of the file):

I can setup the measurement run with this graphical user interface I created:

Data Captured By the Receiver

ROM codes, and 12 bytes of description for each DS18B20 sensor, are stored in the EEPROM of the enclosure's ATMega328P. One temperature sensor is permanently connected to the enclosure, but I can connect as many other sensors as I wish. However, the ROM codes and descriptions for those other sensors must be stored in the EEPROM. When I apply power to the enclosure, we find out what sensors are attached by attempting to read from the scratchpad of each sensor for which we have a ROM code. If the CRC passes, we know the sensor is connected.

I have a full suite of ATmega328P, C++ sketches, for adding, editing, and removing sensor data from the EEPROM. Check it out here. I also wrote my own library for the DS18B20 temperature sensor, and other libraries. Check out my libraries here.

Let's assume I have data for five sensors stored in the EEPROM and sensor's 1, 3, and 5 are connected to the enclosure. I command each sensor, in turn, to make a temperature measurement. When the measurement is available, we assemble 20 bytes of data (data for one sensor). The first byte is always the number of sensors that are connected to the enclosure. For our example, that number is 3. The next to the last byte is the number of the sensor which corresponds to its position in the EEPROM. In our example, that will be 1, 3, or 5. The other bytes correspond to the temperature measurement, high and low alarm temperature, and the sensor description. The last byte is the CRC. We continually make measurements running through each of the connected sensors.

The receiver, continually, looks for valid data. If it recognizes synchronization pulses (see my March 14 post), it sees that the data is valid. It decodes the 20 bytes of Manchester encoded data for the sensor that happened to be transmitting. It may be sensor 1, 3, or 5. It looks at the first byte of data to find the number of sensors in the measurement run. In this case it will be three. An array is initialized whose length is the number of sensors, three in our case, times the number of data bytes per measurement, which is 20. In our example, the array will be 60 bytes long. The data for the first sensor is placed in the array. Next, the receiver code looks for, and captures, data for the next two measurements. This data is appended to the array. Now we have 60 bytes of data. The order of the data may be for sensors 1, 3, and 5; or 3, 5, and 1; or 5, 1, and 3.

After accumulating the 60 bytes, the receiver code looks to see if the Raspberry Pi's python program is requesting the data. If not, the data in the array is overwritten with the next series of measurements. There will be a minimum of one minute between requests from the Pi to the receiver (selected by the user). Depending upon the resolution of the sensor and how power is applied to the sensor (normal or parasitic) the receiver will have those 60 bytes available about every 300ms. to every 2200ms. Therefore, you can see that most of the measurements are lost.

Receiver Code

I have taken another fragment of the C++ code running on the Gertboard's ATmega328P. Visit here to see the entire script.

The code fragment includes the function, "transmit_data()" that looks for a request from the Pi and transfers the data to the Pi. I also included some of the code in "loop()" to illustrate the sequence of events.

To initiate a data transfer cycle, we first look for synchronizing pulses by running line 41 (see blog post of March 14 for this function). We stay here until we find valid synchronization. Once synchronization is confirmed, we move to line 42, the "manchester_data()" function. This will return data for one sensor - the sensor that was transmitting at the time. The code for the "manchester_data()" function can also be found at my March 14 post. If the CRC passes, we now read the first byte of the data to see how many sensors are connected to the enclosure. Now, that we know how many sensors, we can declare the array "xmit_data[]". This array will hold all of the data for all of the sensors for one measurement cycle. After populating "xmit_data[]" with the data from the one sensor we have acquired, we repeat "synchronize()" and "manchester_data()" for each of the other sensors in the run. Now, we have all of the data. Next is to see if the Raspberry Pi has called for the data.

We are now down at line 68 assuming there were no CRC errors. Line 68 takes us to the function "transmit_data()". The function looks to see if there is anything on the ATmega's serial bus (line 15). If so, it is read and put into the array "buff[]" by implementing "Serial.readBytes()". We are looking for one character, and that character is "s". If we find the "s", we place all the data, one byte at a time, onto the serial bus (via the print statement in line 19). If there was no request from the Pi for data, the contents of "xmit_data[]" will be overwritten with new data.

Python Code Running In the Raspberry Pi

The following is a fragment from a rather long program running on the Pi. The program handles setting up the log files for storing the data, sets up the graph parameters, obtains sensor information (description, high and low alarm settings, and resolution), calls the GUI program and incorporates the user responses, makes and records the temperatures, and creates the graphs. The entire program is here. This program calls another python program that handles the graphical user interface. That program is here.

The function "number_of_sensors()" is called once to obtain the sensor information: description, alarms, and resolution. Then, it is called every time the program calls for temperature measurements. When the function exists, if there was valid data, all of the data from all of the sensors will be in the global array "recv_data[]". In addition, the function returns the number of sensors in the run.

"number_of_sensors()" calls the function "retrieve_serial()". "retrieve_serial()" does the actual work of requesting data from the ATmega328P on the Gertboard, and retrieving the response from the Gertboard. When requested by "number_of_sensors()", it clears the serial port of lingering data, writes the character "s" to the serial port, and waits for a response from the Gertboard. It will check every second for data. If it sees no data, or if the length of the data does not equal a multiple of 20 bytes, it keeps looking. If it has not found valid data after 10 seconds, it gives up and returns "0". If valid data is seen, it returns the number of bytes received.

16 X 2 LCD Display Times Square Scroll, Continued

This is a continuation of the prior post about the Times Square Scroll display on a 16 character by 2 line LCD Display.  The entire Python code for that program is at the top of that blog post.  I discussed a code fragment for a simplier version that only displays on the top line of the display.  That fragment replaced lines 49 through 75.  It's now time to discuss the two line version.  So that the reader will not have to go back and forth between this post and the last post, I have repeated the code fragment for the top line only display and made a code fragment that corresponds to lines 49 through 75 of the complete program.

Code Fragment for Top Line Only Display

Code Fragment for Two Line Display

Recall in the prior post I showed that it is possible to have a garbled display of the text where previously entered characters on the other display line, and newly entered characters both get displayed.  Recall Fig. 5 from the last post.

There are two opportunities for this to occur with the two line display.  We have the end of the address space situation discussed in the last post.  Recall, if we don't take corrective action, the next character will be displayed on the wrong display line.  That situation will be solved in nearly the same manner as we solved it before.

The end of the address space situation is solved in the four lines, 12 to 15, in the top line only display.  The four lines, 17 to 20, handle this in the two line display.  The code is very similar but the two line display must deal with characters on both lines.  The two line version simply subtracts 28h from the address counter if the address counter reaches 28h or 68h.  I'm going to explain line 18 further in my next post which talks about hexadecimal numbers.

Now let's discuss the second cause of a garbled display.  When we have completed the first pass through the text on the top line, we move down to the bottom line.  However, the characters that we put into the top line memory are still in that memory.  Eventually, as we add more characters to the bottom line, and the Visible Memory window continues to move, the characters that are in the top line memory start to become visible.  When we finish the bottom line, we move to the top line and the old bottom line characters evenually start to show up.  This goes on and on until we push the switch to stop the program from running.

I solve that problem by overwriting the old characters with spaces.  Evety time I finish writing a character to the display, I erase a character before I write the next character.  I could not allow the display to shift automatically after I write a character.  Code line 1 of the top line only version does not appear in the two line version.  After I write a character (code line 11), I write a space to the same address position on the other display line (code lines 12 and 13).  I then restore the address counter back to the line I was writing characters to, and increment the address counter (code line 14, and increment by adding 81h rather than 80h to the address counter).  Finally, I manually shift the display to the left, which moves the Visible Memory window to the right (code line 15).

For example, if I write a character to memory position 10h on the top line, I write a space to memory position 50h, overwriting any character that was there.  Look at Figures 2,3, or 4 in the last post for reference.

Code line 12 of the two line version will be discussed further in the next post.

16 X 2 LCD Display Times Square Scroll

Here is a more complex program using the LCD display.  Just watch the video to see what the program does:

Here is the python code:

As you can see, the Times Square Scroll program asks the user to input text.  That text enters at the right end of the LCD display, on the top line, and scrolls to the left.  Once the entire text has gone to the display, it starts over on the bottom line.  Once all of the text has appeared on the bottom line it repeats to the top line.  This continues until the user pushes the switch and stops the process.

There is not a great deal of additional code to make the Times Square Scroll program work.  The shortness of the code, however, belies the fact that it is somewhat complex and requires a pretty good understanding of the 44780 display controller.  Remember that all of the code from the file LCD_2X16.py is imported into this program.

The display controller has two internal memories.  One is the character memory that converts an ASCII character to a pattern of dots that correspond to the letter, number, or symbol you see on the display.  It is the other memory, however, that concerns us.  This memory, called the Display Data RAM, contains 80 memory locations (80 eight bit bytes).  When you send a character to the display controller, it goes into this RAM.

The first 40 memory locations are devoted to the top line of the display, and they correspond to memory addresses 00h to 2Fh.  The bottom line's 40 memory locations are found at addresses 40h to 67h.  The data sheet for the display controller uses Hexadecimal notation for memory addresses, so, I will be using Hexadecomal notation too.  While there are 80 memory locations, only a maximum of 32 characters appear on the display.  Obviously, characters in some some of the memory are visible, and some are not.  To make it easier to determine what is visible, we can define a window corresponding to the visible portion of the memory.  Let's call that the Visible Memory window.  Take a look at Fig. 1, below.  It is a graphical representation of Display Data RAM.  I have indicated the Visible Memory window with a red rectangle.  It is important to know that this window can be moved to display characters in different memory locations, but it will always be the same size.

I am going to talk about sending characters to the memory.  Actually, we will be writing the ASCII equivalent of the characters to the memory.  For example, if we want a space, 20h, or 32 decimal will be written.  We'll keep it simple and say we are sending a space to the memory.

If you want a simple display project, like in my previous blog, you would not need to concern yourself with the Display Data RAM and my Visible Memory window.  Figure 1, shows how the memory was utilized in the previous blog.  The Visible Memory window stayed put and there were never any characters placed in the memory locations that were not visible.  The Times Square Scroll, however, makes plenty use of the Visible Memory window.

Figure 1.

There are two other important memory elements within the 44780 controller chip.  The first is the instruction register.  The commands (as opposed to characters) you write to the controller go there.  Pages 23 through 29 of the controller's data sheet explain the commands quite well.  That other important element is the address counter.  The address counter contains the memory location that will receive the next character.  You can write to the address counter if you wish to change where the next character will go - the code does this frequently.  It is important to know that once you write a character into a memory location, the controller will increment (or decremen) the contents of the address counter.  In this way you don't have to keep changing the address counter yourself.  As a matter of fact, you can't stop the address counter from incrementing or decrementing.

Before we look at the code for this two line version of the Times Square Scroll, I have a simpler version that uses the top line only.  The code fragment, below, replaces lines 49 through 75 above.  It's a little easier to understand.

Let's assume the user has asked to send "Now is the time for all good men to come to the aid of their party" to the display.  Recall that I said you can change the contents of address counter, and that is what I do in line 3 and 6.   I set it to the position just to the right of the Visible Memory window and upload it to the display controller.  In line 1, I commanded the controller to make a left shift after I upload each character.  Line 1 also says to increment the address counter after uploading each character.  If I had been writing in a language that reads from right to left, Hebrew, for example, I would have told it to decrement the address counter.  When I start to upload characters, the first character "N" goes to position 10h.  See Figure 2.

Figure 2.

After uploading the "N", the display does a left shift.  Hitachi considers the shift direction from the point of view of the character.  From my point of view, the Visible Window moves one position to the right.  In effect, it moves the "N" one position to the left.  The "N" is now within the Visible Window, so it appears on the display.  See Figure 3.

Figure 3.

When the "o" is uploaded, the address counter has been incremented to 11h (by the controller), which again, is one position to the right of the Visible Window.  Once uploaded, the "o" becomes visible because the controller shifts the Visible Window. See Figure 4.  This process continues as each character is added to the display.

Figure 4.

What happens once the address counter has gotten to the end of the memory space of the top line (address 27h)?  Lines 12 to 15 of the code handle that situation.  If those lines of code were not there, the address counter would move to the beginning of the bottom line.  As more characters are added, characters that were previously uploaded to the top line, and still in memory, would also start to be visible.  Old characters would be seen on the top line while new characters are seen on the bottom line.  Eventually, old and new characters would be seen on both lines and you get a mess like in figure 5.

Figure 5.

Code lines 12 to 15 along with line 3 address that problem.  We keep track of the address counter, and when it reaches the memory address beyond the end of the top tine (28h), we simply set it to the beginning address of the top line, 00h.  Note that the Visible Window will now split in two parts.  See Figures 6 and 7.

I could have avoided the problem in another way.  I could have told the display controller that I had a one line display. I could have written:

lcd_byte(0x20, LCD_CMD)

By saying I had a one line display, I could have dispensed with code lines 12 to 15.  However, I knew the one line display was going to be a precursor to my ultimate two line Times Square Scroll.  The two line version, also has code similar to lines 12 to 15.

Figure 6.

Figure 7.

That's enough to absorb for now. I'm going to save the rest of the discussion of the Times Square Scroll for the next blog entry.

16 X 2 LCD Input Text and Display

Here is a more practical program that asks you to input text and then displays that text on the LCD display.  The challenge here was to parse the input text so that words were not broken up.  Click on the video to see it work:

Here is python code for this program:

I apologize that the name of the program in the video (LCD_Module_1,py) does not match the name in the Gist file (LCD_Disploy2Lines.py). Somewhere along the way, after I made the video, I changed the file name to make the name more meaningful, at least to me.

The program does not take into account individual words of more that 16 characters in length.

Notice line 16 of the code. That makes LCD_2X16.py (discussed in the last post) part of this program.

16 X 2 LCD Base Code

I am finally getting to the code.  I will spread this over three posts.  The first code will be the base module that will be imported into python programs for two other projects.  The base module can, of course, be run by itself. If it is run by itself, the LCD will display what you see in the photograph in the header of this blog.  

I did not originate the base module.  I based it on the code found at Matt Hawkins blog (referenced previously).  I added to it, made some modifications, and made it more suitable to be imported into other projects.  Matt was not the originator of the code either.  I believe the original code was modified from code for the Audunio and appears in Micky Skyler's Adafruit tutorial, "Drive a 16x2 LCD with Raspberry Pi" that is referenced in the previous post.  That tutorial presents a python class for the display controller called Adadruit_CharLCD.py.  This code is quite comprehensive and I believe others took what they needed from it for their code.  This is my version of that code, my base module:

You will need to have RPI.GPIO installed to control the GPIO pins on the PI.  I believe some distributions already include it.  Micky Skyler's Adafruit tutorial covers how to get it if you need it.  

Lines 23 and 24 are probably for my situation only.  More on this in my next post.  

If you want to understand the code you have to read it along with the Hitachi data sheet for the HD4478U.  Pages 24 and 25 have the pertinent information.  

Lines 40 through 42 of the code are particularly interesting (and I can't take any credit for this code). The HD44780 must be initialized before it can be used.  See Figure 24 on page 46 for the sequence of events to initialize the chip.  There are timing requirements between steps like "Wait for more than 4.1ms".  The PI is slow enough that we don't have to worry about the timing.  

The initialization starts (line 40 of the code) with the controller in 8 bit mode, not four bit mode.  The first two commands have DB7 - DB4 (data bus pins) set to 0011 which corresponds to hex 3, so the first two hex 3's are sent, one hex 3 at a time in line 40. The next two commands are 0011 and 0010, corresponding to hex 3 and hex 2, which are sent to the controller by line 41.  The next line, 0010 specifies 4 bit operation which is hex 2 (8 bit operation would be 0011), while the following line has N = 1 for a two line display and F = 0 for 5x8 dots.  Therefore the forth command is 1000, or 8 hex. Code line 42 sends those two commands, hex 2 and hex 8, to the controller.  I'll leave the rest of the initialization commands for you to figure out for yourself.  

For those unfamiliar with hexadecimal numbers, I plan to devote a post to hex, binary and decimal numbers and bit manipulation.

Line 105, if __name__ == '__main__', was something that I did not understand at first.  I saw quite a few questions about it on forums, so I guess I was not the only person who wanted to know.  Why do we need this line at all?  I need it because I intend to import this base module into a couple of other programs for my next posts.  This will save me from copying lines 12 to 102 into the new programs, and you will only see the new code giving a cleaner presentation.  

So how do you interpret line 105?  You can consider __name__ and __main__ to be python interpreter or internal variables.  When you run my base module, LCD_2X16.py, the interpreter assigns '__main__' to __name__.  Since line l05 evaluates to true, the code following that line will run, outputting the text to the display.  However, I have imported LCD_2X16.py into other programs by writing from LCD_2X16 import * in those programs.   If I then run one of those programs, everything from LCD_2X16.py will be part of the calling program.  However, when if __name__ == '__main__' is reached it will compute to false because __name__ now equals 'LCD_2X16', not '__main__' thus the test code will be ignored.  Pretty neat!  if __name__ == '__main__' is only written in the base module, not my other programs.