Using SD As a Data Lookup Table With Arduino

When I plan my projects I often come against the same problem, lack of storage on the humble Arduino. Many of my designs need data to operate for example I am in the middle of designing a kitchen scale that provides information on the products you are weighing, like calories, fat content etc..

The idea is that somebody on a calorie controlled diet could enter the product and the scale would provide the relevant information for that product. The smart scales I have seen involve looking up the product in a catalogue and getting a number to enter into the scales, talk about a chore!

This project needs a look up table for information to be used and displayed.

So I did what most people do, I searched ... Instructables, YouTube, Forums etc..

I could not find any information on reading individual data from a SD card. Every search just brings up "data logger".

Of course there are methods for retrieving data from web servers but I want to have a stand alone unit without Wi-Fi.

Now I would like to share my research with you.

Arduino Uno

SD card module/shield

SD card

Hook up wires

The code is provided, as is, to give an idea of how I go about retrieving data from the SD card.

Start by wiring up your SD card reader module to the Arduino. In my case I follow the standard method used by the built in sketch examples.

SD module -> Arduino

Vcc -> 5v
Gnd -> Gnd
MOSI -> Pin 11
MISO -> Pin 12
Clk -> Pin 13
CS -> Pin 4

If you have a shield you will need to adjust the pins to suit, also be aware some boards run at 3.3v logic levels.

I create my data file on my PC, I pop the SD card into an adapter, use a standard text editor and put each entry on a new line for example:

apple,40
pear,38
grape,39
plum,42
fat free yoghurt,65
.
.
.

In my case there is only two entries "The produce" and "The calories per 100g"
Of course this data is a total work of fiction but it demonstrates the concept.

The file is saved to the SD card and then placed into the SD adapter connected to our Arduino.

The algorithm for retrieving the data to use in our Arduino is as follows:

1st Enter a search term via the serial monitor (eg. "plum˿")
2nd Remove the newline from the end of our entry (eg. "plum")
3rd Open the file (test.txt) and search for our search term
4th If we find the term record the position within the file
5th Move back to the start of the matched data
6th Grab the string data until the comma ("plum")
7th Grab the integer data until the newline ("42")
8th Use the data in our sketch.

Now you can see how to retrieve data you can expand on it to use more fields and just populate more variables or a struct, array etc. and you could expand on it to read more than one entry.

Your text lines would just be formulated like:
search item,value1,value2,value3 ... [newline]
You would then just grab each value and typecast it to the right variable.

In the case of my "scale" project I am using a home brew keyboard, either to allow the user to select a character and scroll through the available produce, or enter a produce to find, not sure which method I will opt for yet.
The advantage of using SD is it is easy to update/modify and is a stable form of storage

The same method can be used for EEPROM but with the standard UNO you only get 1024 bits to work with and writing data to the EEPROM reduces its life cycle.

I hope this is useful food for thought for somebody and stay tuned for progress on the smart scales.

Here is the code (raw) :

/*
  SD card used as look up table

  The connections between SD module and Arduino:-
    
 MOSI -> pin 11 on Arduino Uno/Duemilanove/Diecimila
 MISO -> pin 12 on Arduino Uno/Duemilanove/Diecimila
 CLK -> pin 13 on Arduino Uno/Duemilanove/Diecimila
 CS -> Pin 4 (Dependant upon your SD card shield or module)
*/

// include the SD library:
#include 
#include 


const int chipSelect = 4; // according to your shield/module
File TestFile;
 unsigned long timer;
 char buffer[255];
 char findthis[20];
 bool found=false;
 unsigned int calories,fp;
 String produce;
 
void setup() {
  // Open serial communications and wait for port to open:
  Serial.begin(9600);
  while (!Serial) {
    ; // wait for serial port to connect. Needed for native USB port only
  }
  Serial.print("\nInitializing SD card...");

  if (!SD.begin(SPI_FULL_SPEED, chipSelect)) { // go as fast as your SD will allow.
    Serial.println("initialization failed. Things to check:");
    Serial.println("* is a card inserted?");
    Serial.println("* is your wiring correct?");
    Serial.println("* did you change the chipSelect pin to match your shield or module?");
    while (1); // kill sketch :)
  } else {
    Serial.println("Wiring is correct and a card is present.");
  }
 }

/* 
   ######################### GetData ############################################################### 
   #### function to search our data, on SD card, for matching entry so as to return wanted data #### 
   #### input what we want to look for                                                          ####
   #### return nothing - but could return an index to found data or other usful stuff           #### 
   #################################################################################################
*/
void GetData(char what[255]){
 timer=millis(); //<- i use this to time how long searches take but it can be left out 
 Serial.println("Searching SD"); // friendly messages
  TestFile=SD.open("test.txt",FILE_READ); // The file is created on a pc/mac using the format "product name,calories\n\r" for each line.
  if (TestFile) // ok opened!
   {    
     Serial.println("Test file Opened!");
   } else { 
     Serial.println("Test file failed to open!");
   }
 strcpy(findthis,what); // move our search term into our buffer
 findthis[strlen(findthis)-1]=0;//remove newline/termination as the search wont match with it on the end
 /* Serial.print("Searching for ");
 Serial.println(findthis);
 Serial.println(strlen(findthis)); */ //for debug
 while (TestFile.available()) {
   found=TestFile.find(findthis); // search our file
   if (found) {
    // Serial.println("Found it!"); // debug
    fp=TestFile.position(); // get the file pointer which will be at the end of the matching term 
    fp-=strlen(findthis); // move to the start of our matching term - 
    TestFile.seek(fp);    // we do this by deducting the size of our search term from the file pointer
    produce=TestFile.readStringUntil(44); // now we read until we find the comma
    
    /* ###################################### more data ##############################################
    /* next variable -> variable2=TestFile.readStringUntil(44); // now we read until we find the comma 
       Do this for however many fields you need for your project
    */
    
    calories=TestFile.readStringUntil(13).toInt(); // next we read to the end of the line to get the value
    Serial.print("Produce: ");
    Serial.print(produce);
    Serial.print(" has ");
    Serial.print(calories);
    Serial.println(" calories per 100g");
    found=false; 
    }
   }
  
  TestFile.close(); // close the file
  Serial.print("Finished in "); // report how long the search took, you can
  Serial.print(millis()-timer); // leave this out it is purly for my testing.
  Serial.println(" ms");        // I like to keep tabs on information like this.
}

void loop(void) {
 String user_input;
 bool searchon=false;

 /* For testing I am supplying the search term via the serial monitor but you can
    use user input from another source like an attached keyboard or push buttons etc...
 */
 while(Serial.available()) {
  user_input=Serial.readString();
  user_input.toCharArray(buffer,255);
  searchon=true;
  }

  if (searchon) {
    GetData(buffer);
  }
}