Arduino Cookbook Excerpt: Large Tables of Data in Program Memory

Arduino
Arduino Cookbook Excerpt: Large Tables of Data in Program Memory

Piroulette Console

If you’ve ever made an Arduino-based project involving a large number of text strings, you’ll quickly find the standard-issue Arduino’s 2K of SRAM very limiting. One solution is to save the text in program memory instead, which at 32Kb is much more plentiful.

In the Spring of 2011, I worked on a project with artists Steve Hanson and Elliot Clapp that was part of Apexart’s “Let it end like this” group show, curated by Todd Zuniga. Steve created a generative questionnaire that polled gallery-goers and presented them with their Last Words. An interactive push-button console and LCD was built by Elliot (pictured above) and we built the thing using an inexpensive Arduino variant from Modern Device. The complete project is documented in the Make: Projects wiki.

Steve’s poetic Last Words took up almost 16K of memory, which was stored in the program memory of the Arduino. Below is an excerpt from the Arduino Cookbook by Michael Margolis that explains how to save and access a large table of data in program memory.

17.3 Storing and Retrieving Numeric Values in Program Memory

Problem

You have a lot of constant numeric data and don’t want to allocate this to RAM.

Solution

Store numeric variables in program memory (the flash memory used to store Arduino programs).

This sketch adjusts a fading LED for the nonlinear sensitivity of human vision. It stores the values to use in a table of 256 values in program memory rather than RAM.

The sketch is based on Recipe 7.2; see Chapter 7 for a wiring diagram and discussion on driving LEDs. Running this sketch results in a smooth change in brightness with the LED on pin 5 compared to the LED on pin 3:

/* ProgmemCurve sketch
 * uses table in Progmem to convert linear to exponential output
 * See Recipe 7.2 and Figure 7-2
 */

#include <avr/pgmspace.h>  // needed for PROGMEM

// table of exponential values
// generated for values of i from 0 to 255 -> x=round( pow( 2.0, i/32.0) - 1);

const byte table[]PROGMEM = {
   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
   0,   0,   0,   1,   1,   1,   1,   1,   1,   1,   1,   1,   1,   1,   1,   1,
   1,   1,   1,   1,   1,   1,   1,   1,   1,   1,   1,   2,   2,   2,   2,   2,
   2,   2,   2,   2,   2,   2,   2,   2,   2,   2,   3,   3,   3,   3,   3,   3,
   3,   3,   3,   3,   3,   3,   4,   4,   4,   4,   4,   4,   4,   4,   4,   5,
   5,   5,   5,   5,   5,   5,   5,   6,   6,   6,   6,   6,   6,   6,   7,   7,
   7,   7,   7,   8,   8,   8,   8,   8,   9,   9,   9,   9,   9,  10,  10,  10,
  10,  11,  11,  11,  11,  12,  12,  12,  12,  13,  13,  13,  14,  14,  14,  15,
  15,  15,  16,  16,  16,  17,  17,  18,  18,  18,  19,  19,  20,  20,  21,  21,
  22,  22,  23,  23,  24,  24,  25,  25,  26,  26,  27,  28,  28,  29,  30,  30,
  31,  32,  32,  33,  34,  35,  35,  36,  37,  38,  39,  40,  40,  41,  42,  43,
  44,  45,  46,  47,  48,  49,  51,  52,  53,  54,  55,  56,  58,  59,  60,  62,
  63,  64,  66,  67,  69,  70,  72,  73,  75,  77,  78,  80,  82,  84,  86,  88,
  90,  91,  94,  96,  98, 100, 102, 104, 107, 109, 111, 114, 116, 119, 122, 124,
 127, 130, 133, 136, 139, 142, 145, 148, 151, 155, 158, 161, 165, 169, 172, 176,
 180, 184, 188, 192, 196, 201, 205, 210, 214, 219, 224, 229, 234, 239, 244, 250
};

const int rawLedPin  = 3;           // this LED is fed with raw values
const int adjustedLedPin = 5;       // this LED is driven from table

int brightness = 0;
int increment = 1;

void setup()
{
  // pins driven by analogWrite do not need to be declared as outputs
}

void loop()
{
  if(brightness > 254)
  {
     increment = -1; // count down after reaching 255
  }
  else if(brightness < 1)
  {
    increment =  1; // count up after dropping back down to 0
  }
  brightness = brightness + increment; // increment (or decrement sign is minus)

  // write the brightness value to the LEDs
  analogWrite(rawLedPin, brightness);  // this is the raw value
  int adjustedBrightness = pgm_read_byte(&table[brightness]);  // adjusted value
  analogWrite(adjustedLedPin, adjustedBrightness);

  delay(10); // 10ms for each step change means 2.55 secs to fade up or down
}

Discussion

When you need to use a complex expression to calculate a range of values that regularly repeat, it is often better to precalculate the values and include them in a table of values (usually as an array) in the code. This saves the time needed to calculate the values repeatedly when the code runs. The disadvantage concerns the memory needed to place these values in RAM. RAM is limited on Arduino and the much larger program memory space can be used to store constant values. This is particularly helpful for sketches that have large arrays of numbers.

At the top of the sketch, the table is defined with the following expression:

  const byte table[]PROGMEM = {
  0, . . .


PROGMEM tells the compiler that the values are to be stored in program memory rather than RAM. The remainder of the expression is similar to defining a conventional array (see Chapter 2).

The low-level definitions needed to use PROGMEM are contained in a file named pgmspace.h and the sketch includes this as follows:

  #include <avr/pgmspace.h>


To adjust the brightness to make the fade look uniform, this recipe adds the following lines to the LED output code used in Recipe 7.2:

  int adjustedBrightness = pgm_read_byte(&table[brightness]);
  analogWrite(adjustedLedPin, adjustedBrightness);


The variable adjustedBrightness is set from a value read from program memory. The expression pgm_read_byte(&table[brightness]); means to return the address of the entry in the table array at the index position given by brightness. Each entry in the table is one byte, so another way to write this expression is:

  pgm_read_byte(table + brightness);


If it is not clear why &table[brightness] is equivalent to table + brightness, don’t worry; use whichever expression makes more sense to you.

Another example is from Recipe 6.5, which used a table for converting an infrared sensor reading into distance. Here is the sketch from that recipe converted to use a table in program memory instead of RAM:

 /* ir-distance_Progmem sketch
  * prints distance & changes LED flash rate
  * depending on distance from IR sensor
  * uses progmem for table */

  #include <avr/pgmspace.h> // needed when using Progmem

  // table entries are distances in steps of 250 millivolts
  const int TABLE_ENTRIES = 12;
  const int firstElement = 250; // first entry is 250 mV
  const int interval = 250; // millivolts between each element
  // the following is the definition of the table in Program Memory
  const int distanceP[TABLE_ENTRIES] PROGMEM = {
    150,140,130,100,60,50, 40,35,30,25,20,15 };

  // This function reads from Program Memory at the given index
  int getTableEntry(int index)
  {
    int value = pgm_read_word(&distanceP[index]);
    return value;
  }


The remaining code is similar to Recipe 6.5, except that the getTableEntry function is used to get the value from program memory instead of accessing a table in RAM. Here is the revised getDistance function from that recipe:

  int getDistance(int mV)
  {
    if( mV > interval * TABLE_ENTRIES )
      return getTableEntry(TABLE_ENTRIES-1); // the minimum distance
    else {
      int index = mV / interval;
      float frac = (mV % 250) / (float)interval;
      return getTableEntry(index) - ((getTableEntry(index) -
        getTableEntry(index+1)) * frac);
    }
  }


In the Maker Shed:
Makershedsmall

Arduino Cookbook
Create your own robots, toys, remote controllers, alarms, detectors, and more with Arduino and this guide. Arduino lets artists and designers build a variety of amazing objects and prototypes that interact with the physical world. With this Cookbook, you can dive right in and experiment, thanks to more than a hundred tips and techniques, no matter what your skill level. Here you’ll find the examples and advice you need to begin, expand, and enhance your projects right away.

14 thoughts on “Arduino Cookbook Excerpt: Large Tables of Data in Program Memory

  1. Anonymous says:

    The Arduino Cookbook code has a bug in it.  The first line in the loop function should read:

    if(brightness == 255)
    As originally written, the “brightness” variable would increment up to a high of 256.  It would then be used as an index into the “table” array, which only goes up to an index of 255 (it has 256 members, so the indexes are 0 => 255).  For you non-programmers, reading data beyond the bounds of your array is bad!  The next time through the loop it would take the (brightness > 255) branch of the if and finally start decrementing.

    I don’t have the cookbook, so I don’t know if this bug is in it or if it has already been fixed.  Free copy for fixing errors? :)

    1. Shawn Wallace says:

      You’re right; the boundary condition should compare to 255 instead of 256. The result will be reading an unintended brightness value every full cycle…I’ve submitted an errata report and fixed the code in the blog!

    2. Shawn Wallace says:

      You’re right; the boundary condition should compare to 255 instead of 256. The result will be reading an unintended brightness value every full cycle…I’ve submitted an errata report and fixed the code in the blog!

  2. Anonymous says:

    The book also discusses how to read strings from the program memory as well as writing and reading from the EEPROM memory.

    I find it to be a good resource in most cases.

    Also ProfessorStupid… You are correct there is a bug. I think though that a better solution would be to move the line brightness= brightness + increment; down to the bottom of the loop() function.

    As it stands it skips out on the 0th element when it first runs.

    Better yet just use a modulo and you never have to worry about it such as:

    brightness = (brightness +1 ) % 256;

    That way the range of brghtness is always from 0 to 255. When brightness +1 = 256 the modulo (remainder) will be 0.

    1. Anonymous says:

      The modulo is elegant – but wrong!  You’ll jump from 255 to 0 instead of going smoothly back down as the code intends.

      1. Anonymous says:

        Zoinks and double zoinks… Yes you are right I forgot the decrement. My humble apologies.

      2. Anonymous says:

        Zoinks and double zoinks… Yes you are right I forgot the decrement. My humble apologies.

Comments are closed.

Discuss this article with the rest of the community on our Discord server!
Tagged
Shawn Wallace

Shawn Wallace is a MAKE contributor, artist, programmer, and editor living in Providence, R.I. He designs open hardware kits at Modern Device and organized the Fab Academy at the Providence Fab Lab. He makes iPhone synthesizers with the Fluxama collective and is a member of the SMT Computing Society.

View more articles by Shawn Wallace

ADVERTISEMENT

Maker Faire Bay Area 2023 - Mare Island, CA

Escape to an island of imagination + innovation as Maker Faire Bay Area returns for its 15th iteration!

Buy Tickets today! SAVE 15% and lock-in your preferred date(s).

FEEDBACK