Skip to content
This repository
Browse code

Cleaning up directory structure.

Filing firmware and hardware into their own directories.
  • Loading branch information...
commit dcc9b26b701751f04dc2e752b4fad7bb03ce69c0 1 parent 8f5db46
Nathan Seidle nseidle authored
BIN  FTDI-Connections.psp
Binary file not shown
0  OpenLog.pdf → OpenLog-Schematic.pdf
File renamed without changes
960 OpenLog_v3/Changes.txt → firmware/OpenLog_v3/Changes.txt
... ... @@ -1,481 +1,481 @@
1   - 12-3-09
2   - SparkFun Electronics 2012
3   - Nathan Seidle
4   -
5   - OpenLog hardware and firmware are released under the Creative Commons Share Alike v3.0 license.
6   - http://creativecommons.org/licenses/by-sa/3.0/
7   - Feel free to use, distribute, and sell varients of OpenLog. All we ask is that you include attribution of 'Based on OpenLog by SparkFun'.
8   -
9   - OpenLog is based on the work of Bill Greiman and sdfatlib: http://code.google.com/p/sdfatlib/
10   -
11   - OpenLog is a simple serial logger based on the ATmega328 running at 16MHz. The ATmega328
12   - is able to talk to high capacity (larger than 2GB) SD cards. The whole purpose of this
13   - logger was to create a logger that just powered up and worked. OpenLog ships with an Arduino/Optiboot
14   - 115200bps serial bootloader running at 16MHz so you can load new firmware with a simple serial
15   - connection.
16   -
17   - OpenLog automatically works with 512MB, 1GB, 2GB, 4GB, 8GB, and 16GB microSD cards. We recommend FAT16 for 2GB and smaller cards. We
18   - recommend FAT32 for 4GB and larger cards.
19   -
20   - OpenLog runs at 9600bps by default. This is configurable to 2400, 4800, 9600, 19200, 38400, 57600, and 115200bps. You can alter
21   - all settings including baud rate and escape characters by editing config.txt found on OpenLog.
22   -
23   - Type '?' to get a list of supported commands.
24   -
25   - During power up, you will see '12<'. '1' indicates the serial connection is established. '2' indicates
26   - the SD card has been successfully initialized. '<' indicates OpenLog is ready to receive serial characters.
27   -
28   - Recording constant 115200bps datastreams are supported. Throw it everything you've got! To acheive this maximum record rate, please use the
29   - SD card formatter from : http://www.sdcard.org/consumers/formatter/. The fewer files on the card, the faster OpenLog is able to begin logging.
30   - 200 files is ok. 2GB worth of music and pictures is not.
31   -
32   - To a lower dir, use 'cd ..' instead of 'cd..'.
33   -
34   - Only standard 8.3 file names are supported. "12345678.123" is acceptable. "123456789.123" is not.
35   -
36   - All file names are pushed to upper case. "NewLog.txt" will become "NEWLOG.TXT".
37   -
38   - Type 'set' to enter baud rate configuration menu. Select the baud rate and press enter. You will then
39   - see a message 'Going to 9600bps...' or some such message. You will need to power down OpenLog, change
40   - your system UART settings to match the new OpenLog baud rate and then power OpenLog back up.
41   -
42   - If you get OpenLog stuck into an unknown baudrate, there is a safety mechanism built-in. Tie the RX pin
43   - to ground and power up OpenLog. You should see the LEDs blink back and forth for 2 seconds, then blink
44   - in unison. Now power down OpenLog and remove the RX/GND jumper. OpenLog is now reset to 9600bps.
45   -
46   - Please note: The preloaded Optiboot serial bootloader is 0.5k, and begins at 0x7E00 (32,256). If the code is
47   - larger than 32,256 bytes, you will get verification errors during serial bootloading.
48   -
49   - STAT1 LED is sitting on PD5 (Arduino D5) - toggles when character is received
50   - STAT2 LED is sitting on PB5 (Arduino D13) - toggles when SPI writes happen
51   -
52   - LED Flashing errors @ 2Hz:
53   - No SD card - 3 blinks
54   - Baud rate change (requires power cycle) - 4 blinks
55   -
56   - OpenLog regularly shuts down to conserve power. If after 0.5 seconds no characters are received, OpenLog will record any unsaved characters
57   - and go to sleep. OpenLog will automatically wake up and continue logging the instant a new character is received.
58   -
59   - 1.55mA idle
60   - 15mA actively writing
61   -
62   - Input voltage on VCC can be 3.3 to 12V. Input voltage on RX-I pin must not exceed 6V. Output voltage on
63   - TX-O pin will not be greater than 3.3V. This may cause problems with some systems - for example if your
64   - attached microcontroller requires 4V minimum for serial communication (this is rare).
65   -
66   - OpenLog has progressed significantly over the past three years. Please see Changes.txt or GitHub for a full change log.
67   -
68   -
69   - v1.1
70   - Adding better defines for EEPROM settings
71   - Adding new log and sequential log functions.
72   -
73   - Code is acting very weird with what looks to be stack crashes. I can get around this by turning the optimizer off ('0').
74   - Found an error : EEPROM functions fail when optimizer is set to '0' or '1'.
75   - sd-reader_config.h contains flag for USE_DYNAMIC_MEMORY
76   -
77   - Looks like tweaking the optimization higher causes the asm("nop"); to fail inside the append_file routine. Changing this to
78   - delay_us(1); works.
79   -
80   - I have a sneaking suspicion that I was having buffer overrun problems when defining the input_buffer at 1024 bytes. The
81   - ATmega328 should have enough RAM (2K) but weird reset errors were occuring. With the buffer at 512bytes, append_file runs
82   - just fine and is able to log at 115200 at a constant data rate.
83   -
84   - Added file called 'lots-o-text.txt' to version control. This text contains easy to scan text to be used for full data
85   - rate checking and testing.
86   -
87   -
88   - v1.2
89   - ringp added:
90   - Adding support for splitting command line parameters into arguments
91   - Adding support for reading files sequencially
92   - read <filename> <start> <length>
93   - New log now for sequencial log functions supports 0-65535 files
94   - Adding support for wildcard listing or deletion of files
95   - ls <wildcard search>
96   - rm <wildcard delete>
97   -
98   - Really great additions. Thanks ringp!
99   -
100   - Nate added error testing within newlog()
101   - Checks to see if we have 65534 logs. If so, error out to command prompt with "!Too many logs:1"
102   -
103   -
104   - v1.3
105   - Added sd_raw_sync() inside append_file. I believe this was why tz1's addition of the timeout buffer update feature
106   - was not working. Auto buffer update now working. So if you don't send anything to OpenLog for 5 seconds,
107   - the buffer will automatically record/update.
108   -
109   - Need to create 'Testing' page to outline the battery of tests we need to throw at any OpenLog after a firmware
110   - submission and update is complete.
111   -
112   - Testing
113   - create 40,000 logs
114   -
115   - Record at full speed:
116   - Run at full 115200, load lotsoftext.txt and verify no characters are dropped.
117   -
118   - Detect too many logs:
119   - Create new log at 65533 (do this by editing 'zero' function to set EEPROM to 0xFF and oxFD)
120   - and power cycle. Verify unit starts new log. Power cycle and verify unit errors out and drops to command prompt.
121   -
122   - Record buffer after timeout:
123   - Create new log. Type 20 characters and wait 5 seconds. Unit should auto-record buffer. Power down unit.
124   - Power up unit and verify LOG has 20 characters recorded.
125   -
126   -
127   - v1.4
128   - Added exit options to the two menus (set and baud)
129   - Also added display of current settin to two menus (Ex: "Baud currently: 57600bps")
130   -
131   - Added '!' infront of 'error opening'. This pops up if there is an error while trying to append
132   - to a freshly created new log (ex: LOG00548.txt is created, then errors out because it cannot append).
133   - '!' was added so that user can parse against it.
134   -
135   - Replicated logging errors at 57600 using 5V Arduino
136   - Unit would systematically glitch during logging of 111054 bytes
137   -
138   - Increasing buffer to 1000 characters caused URU error.
139   - URU: Unit Resets Unexpectedly
140   -
141   - To recreate URU error. Type "append ". Include the space. If you get "!error opening", then things are
142   - fine. If you get "!error opening#" where # is a weird character, then type 'ls' and the unit will
143   - unexpectedly reset (URU error). I believe this is attributed to a memory overrun somewhere in the
144   - FAT system.
145   -
146   - Changed buffer size to 900 and declared the character buffer as volatile
147   - #define BUFF_LEN 900
148   - volatile char input_buffer[BUFF_LEN];
149   -
150   - This increase to the buffer allows for clean logging of 444055 bytes with no URU errors.
151   -
152   - Experimenting with Scott's SD cards (customer gave cards on loan for recreating logging errors):
153   - Card with single ~740mb file produces errors when trying to open/append to new log.
154   - Card with less stuff on it logs full 444055 bytes correctly.
155   -
156   -
157   - v1.5
158   - Added 4800bps and 19200bps support
159   -
160   - Added power saving features. Current consumption at 5V is now:
161   - In default append mode:
162   - 6.6/5.5mA while receiving characters (LED On/Off)
163   - 2.1mA during idle
164   - In command mode: 3.2/2.1mA (LED On/Off)
165   -
166   - So if you're actively throwing characters at the logger, it will be ~6mA. If you send the logger
167   - characters then delay 5-10 seconds, current will be ~2.5mA. (Unit records the characters in the buffer
168   - and goes into idle more if no characters are received after 5 seconds)
169   -
170   - These power savings required some significant changes to uart.c / uart_getc()
171   -
172   -
173   - v1.51 check_emergency_reset, default break character is ctrl+z 3 times, example Arduino sketch
174   -
175   - Added function from mungewell - check_emergency_reset. This has improved testing of the RX pin.
176   - There was potential to get a false baud reset. There is still a chance but it's now much less likely.
177   -
178   - If OpenLog is attached to a Arduino, during bootloading of the Arduino, ctrl+z will most likely be sent
179   - to the Arduino from the computer. This character will cause OpenLog to drop to command mode - probably
180   - not what we want. So I added user selectable character (ctrl+x or '$' for example) and I added
181   - user selectable number of escape characters to look for in a row (example is 1 or 2 or 3, '$$$' is a
182   - common escape sequence). The default is now ctrl+z sent 3 times in a row.
183   -
184   - Added an example Arduino sketch (from ScottH) to GitHub so that people can easily see how to send characters to
185   - OpenLog. Not much to it, but it does allow us to test large amounts of text thrown at OpenLog
186   - at 57600bps.
187   -
188   -
189   - v1.6 Adding config file.
190   -
191   - What happens if I reset the system by pulling RX low, but the config file has corrupt values in it?
192   -
193   - If config file has corrupt values in it, system will default to known values 9600/ctrl+z/3/newlog
194   -
195   - If config file is empty, system resets to known values
196   -
197   - After some massive testing, and lots of code to check for illegal states, it looks to be pretty stable.
198   - The only problem is that we're running out of RAM. The buffer had to be decreased from 900 bytes
199   - to 700 bytes to facilitate all the config file checking. Testing at 57600bps, unit runs very well
200   - over 40kb test file on straight RS232 connection. That's pretty good. Testing at 115200 on straight
201   - connection, unit will drop a buffer every once and a while. Not great, but not much we can do if the
202   - SD card times out for ~150ms while it's writing.
203   - 8 bits to the byte plus a start/stop bit = 10 bits per byte
204   -
205   - @ 9600bps = 960 bytes per second. Buffer will last for 729ms
206   - @ 57600bps = 5760 bytes per second. Buffer will last for 121ms
207   - @ 115200bps = 11520 bytes per second. Buffer will last for 60.7ms
208   -
209   - So if the SD card pauses for more than 60ms, 115200 will have lost data, sometimes. All other baud rates
210   - should be covered for the most part.
211   -
212   - SD cards with larges amounts of data will have increased pause rates. Always use a clean card where possible.
213   -
214   -
215   - v1.61 Small PCB change. Fixed version # in help menu.
216   -
217   - Fixed the firmware version in the help menu to v1.61.
218   -
219   - Updated Eagle files to Eagle v5.9. Fixed weird airwire. Changed D1 LED from Green to Blue.
220   - Will only affect new production after 4-28-10.
221   -
222   - Closed some tickets and wrote some more example Arduino code:
223   - http://forum.sparkfun.com/viewtopic.php?t=21438
224   -
225   -
226   - v2.0 - 25986 bytes out of 30720
227   - Welcome to version 2! We've moved from Roland Riegel's FAT library to Bill Greiman's sdfatlib. OpenLog now works with SD cards
228   - up to 16GB (that is the current largest microSD card we can get our hands on). OpenLog automatically detects and works with FAT16/FAT32
229   - file systems. It also automatically works with normal SD cards as well as SDHC cards.
230   -
231   - Almost all the previous commands have been ported from v1 to v2. The current commands that do not work:
232   - cd.. - does not work. You can change to an upper directory, but you cannot navigate back down the tree.
233   - cat - this command was depricated. HEX printing is now done with the 'read' command. We've added a 5th argument to select between ASCII and HEX printing.
234   - Wild cards do not yet work. So rm and ls do not have wild cards enabled - yet. Help us out!
235   -
236   - Porting OpenLog to work directly under Arduino to work with sdfatlib (http://code.google.com/p/sdfatlib/) by Bill Greiman.
237   -
238   - sdfatlib intrinsically supports FAT16, FAT32 as well as SD and HCSD cards. In a word, it's amazing.
239   -
240   - Needs to be done:
241   - Done - Get config file reading/loading working
242   - Document config file in wiki: if no config file is found, current system settings are used. If config is found, system switches to settings found in file. If system settings are changed, then config file is changed and system uses new settings immediately.
243   - Done - We don't want to constantly record a new config file on each power on. Only record when there is a change.
244   - Get cd.. working
245   - Seperate OpenLog_v2 into multiple files
246   - Re-test too many log files created in the newlog setting - 65535. Potentially create thousands of files and see how sdfatlib handles it.
247   - Done - Test sequential logging.
248   - Get wild card support working for ls and rm
249   - Get backspace working
250   - Test rm with directories, and directory wildcards? Maybe not.
251   - Get power save working
252   - Test compile on a computer that doesn't have WinAVR
253   -
254   - Test commands:
255   - new - works, also in sub dirs
256   - append - works, also in sub dirs
257   - rm - works, but does not yet support wild cards.
258   - md - works, also in sub dirs
259   - cd - you can change up to a sub-directory, but you cannot navigate back down the tree. The current work around is to type 'init'. This will re-init the card and set the directory back to root.
260   - ls - works pretty well but does not yet support wild cards. Shows directories, files, and file sizes. Would be cool if it listed dirs first.
261   - read - works well. Tested 0, 1, 2, 3, 4, and 5 arguments (included and left off). Fails gracefully. Now prints in HEX as well!
262   - size - works well
263   - disk - works well, prints more information than ever before!
264   - init - works well
265   - sync - works, but not really tested
266   - cat - I've decided to drop this command. You can now print in hex using the read command and using a 5th argument as '1' for ASCII (default) and '2' for HEX.
267   -
268   - set - works well
269   - baud - works well
270   -
271   -
272   - v2.1 - Power save not working. Fixed issue 35. Dropping characters at 57600bps.
273   - 26058 bytes out of 30720
274   - Fixed a bug found by Salient (Issue 35). User settings where declared at chars which allowed them to be signed. If a user went from old firmware, to v2,
275   - the safety checks would fail because the settings would be read at -1 instead of 255. Declaring user settings as byte fixed issue.
276   -
277   - Added "a) Clear user settings" to set menu. This allows us to completely wipe an OpenLog (user settings and config file) to see how it will respond
278   - to future firmware changes.
279   -
280   - Improved the file 'size' command.
281   -
282   - Sequential logging is tested and works.
283   -
284   - Receive testing: Using the Test_Sketch found on Github, I am testing the receive reliability at different UART speeds.
285   - We need to test a lot of received data. At 57600, 115200, and both from an Arduino (lots of time in between characters becuase of software overhead)
286   - and from a raw serial port (almost no time in between characters). I am hoping to make sdfatlib hiccup at 115200, full speed, across a 1MB file. If
287   - I can make it fail, then we can start to increase the buffer size and look at how much RAM sdfatlib has left open for the buffer.
288   -
289   - 9600bps from Arduino works fine
290   - 57600bps from Arduino drops characters
291   - 115200 from Arduino drops characters
292   -
293   - It seems that sdfatlib takes longer to write to the SD card than the original file system from Robert Reigel. I'm thinking perhaps
294   - we should create a version of OpenLog firmware that is just sequantial logging, no fancy system menus... It might open up some RAM.
295   -
296   - If only we could get control of the UART from Arduino's clutches, we could probably handle the ring buffer much better. Not sure how to handle UART
297   - interrupts without tearing out HardwareSerial.cpp.
298   -
299   - Added README to the Test sketch. Added 115200bps to test sketch.
300   -
301   -
302   - v2.11 Tested with 16GB microSD. Fixed some general bugs. Lowered power consumption.
303   -
304   - 26136 bytes out of 30720
305   -
306   - Fixed issue 30. I added printing a period ('.') for non-visible ASCII characters during a 'read' command. This cleans up the output a bit. HEX
307   - printing is still available.
308   -
309   - Fixed issue 34. When issuing a long command such as "read log00056.txt 100 200 2" (read from spot 100 to spot 200 and print in HEX), the
310   - command shell would die at 24 spots. I increased both the general_buffer and 'buffer' in the command shell from 24 to 30. The limit is now
311   - 30 characters, so "read log00056.txt 100 20000 2" is allowed.
312   -
313   - Works with a 16GB microSD card! High volume test: loaded 16GB card with 5GB of data. Basic serial tests work. When running at 57600, there
314   - is an odd delay. I think it has to do with the file system doing an initial scan for an available cluster. Takes 2-3 seconds before OpenLog
315   - returns to normal. This can cause significant data loss.
316   -
317   - Fixing power management in v2. Power down after no characters for 3 seconds now works. Unit drops to 2.35mA in sleep. 7.88mA in sitting
318   - RX mode (awake but not receiving anything). There is still a weird bug where the unit comes on at 30mA. After sleep, it comes back at the
319   - correct 7-8mA. Is something not getting shut off?
320   -
321   -
322   - v2.2 Modified append_file() to use a single buffer. Increased HardwareSerial.cpp buffer to 512 bytes.
323   -
324   - More testing at 57600. Record times look to be 2, 5, and zero milliseconds when doing a record. This means that the split buffer doesn't
325   - really make a difference? There are some records that take 150ms, 14ms, etc. At 57600bps, that's 7200 bytes/s, 138us per byte. With a 150ms
326   - pause, that's 1,086 bytes that need to be buffered while we wait... Grrr. Too many.
327   -
328   - I re-wrote the append_file function to use only one buffer. This allows us to more quickly pull data from the hardware serial buffer. Hardware
329   - serial buffer has to be increased manually to 512. This file (hardwareserial.cpp) is stored in the Arduino directory. With testing,
330   - it seems like recording is working more solidly at 57600bps. But now I'm seeing glitches at 19200bps so more testing is required before we
331   - make this the official OpenLog release.
332   -
333   - Moved input_buffer into within the append function. Attempting to shave bytes of RAM.
334   -
335   -
336   - v2.21 ringp fork brought in. Wildcard remove and list commands now work. Remove directory now works! Change directory up/down the tree works again.
337   -
338   - 28440 bytes used of 30720.
339   -
340   - ringp brought back many great commands! Thanks ringp!
341   - rm LOG*.* works
342   - ls *.TXT works
343   - cd .. works again
344   - ls now correctly shows directories first and files following the directories.
345   -
346   - To remove a directory, you have to navigate into that directory. For example:
347   - >cd TEMP (you are now in TEMP directory)
348   - >rm -f TEMP (you are now removing TEMP, and you will be pushed back down one level of the tree)
349   - >ls (shows files and directories where TEMP directory used to be, but TEMP directory should be gone)
350   -
351   - ringp added new commands:
352   - efcount: gives the number of files in the current directory. Example: "efcount" returns "count|3". There are 3 files in the current directory.
353   - efinfo <spot>: gives info about the file in <spot>. Example: "efinfo 2" reports "LOG00588.TXT|45". File number 2 is named LOG00588.TXT and is 45 bytes in size.
354   - verbose <"on"|"off">: sets whether command errors are verbose (long winded) or just the "!" character. Example: "verbose off" sets verbose off. Then if a
355   - command like "blah" is received, then only "!>" is seen from the command line interface. This makes it easier for embedded systems to recognize there
356   - was an error. This setting is not recorded to EEPROM.
357   -
358   -
359   - v2.3 Migrated to v10.10.10 of sdfatlib. Moved to inline RX interrupt and buffer. Improved the ability to receive a constant stream at 57600bps.
360   -
361   - 27334 bytes out of 30720.
362   -
363   - Remember - we had to completely butcher HardwareSerial.cpp so a normal Arduino installation will not work.
364   - C:\arduino-xxxx\hardware\arduino\cores\arduino\HardwareSerial.cpp
365   -
366   - I removed the receive interupt handler from HardwareSerial.cpp so that we can deal directly with USART_RX_vect in the main code. This allows
367   - us to return to the way OpenLog used to operate - by having one buffer split in half. Once one half fills, it is recorded to SD while the other
368   - half fills up. This dramatically decreases the time spent in function calls and SD recording, which leads to many fewer characters dropped.
369   -
370   - The change log at the top of the main code got so large I've moved it to a version controlled "Changes.txt" file.
371   -
372   - By making all these changes, I have broken many things. Ringp - I could use your help here. I apologize for stomping on so much of your work. I was not
373   - good enough to figure out how to re-link from the old function calls to the new sdfatlib setup.
374   -
375   - Backspace is broken - ringp, I saw this fix in one of your commits, but looking at the code, I don't see how it is supposed to work. Either way, we still
376   - get 0x08 when trying to backspace.
377   -
378   - New sdfatlib doesn't have SdFat.cpp so fileInfo doesn't work. These function calls are marked with //Error
379   -
380   - I have chosen to dis-allow deprecated functions:
381   - #define ALLOW_DEPRECATED_FUNCTIONS 0
382   - This forced some trivial changes to the SD write and new file creation function calls. I believe we've successfully migrated to the new version of sdfatlib.
383   -
384   - In the command_shell / read_line function : It may be better to pull directly from the UART buffer and use the RX interrupt. For now, we brute force it.
385   -
386   - Because of all these changes, I need to re-test power consumption. For now, I'm confident it's low enough.
387   -
388   - Testing with 512 buffer array size
389   - 1GB @ 57600 - dropped very little out of 3 tests
390   - 1GB @ 115200 - dropped very little out of 2 tests
391   - 8GB @ 57600 - Formatted using the sd formater (32k byte allocation size). Dropped nothing.
392   - 8GB @ 115200 - dropped very little, dropped none
393   - 16GB w/ Lots of files @ 57600 - Drops the first part of the file because of start up delay?
394   - 16GB w/ Lots of files @ 115200
395   -
396   - 1024 array size (and 800) does not run
397   -
398   - Testing with 700 buffer array size
399   - 1GB @ 57600 - 110300 out of 111000 bytes, 110300/111000,
400   - 1GB @ 115200 - 111000/111000!, 109600/111000
401   - 8GB @ 57600 - 109000/111000, 111000/111000!,
402   - 8GB @ 115200 - 111000/111000!, 111000/111000!,
403   - 16GB w/ Lots of files @ 57600 - 85120/111000, 85120/111000
404   - 16GB w/ Lots of files @ 115200 - 56420 (but once it got going, log looks good). 56420.
405   -
406   - I am seeing long delays on cards with lots of files. In the above tests, the 16GB test card is a good example. It has 2GB worth of random files in a sub directory.
407   - After OpenLog boots, goes to '12<'. After I send ~500 characters OpenLog freezes for a couple seconds, then returns to normal, very fast, operation. During
408   - that down time, I believe sdfatlib is searching for an open cluster. The odd thing is that after the cluster is established (after the initial down time) OpenLog
409   - performs excellently. I am hoping to create a faux file or pre-write and save to the file or some how get this allocation done before we report the
410   - '12<' indicating we are ready. That way, if a card does fill up, as long as the host system is checking for '<', it doesn't matter how long it takes
411   - sdfatlib to find the next free cluster.
412   -
413   - You can see that we drop 700 bytes at a time. That's a bit odd - I'd expect to drop half or 350 at a time.
414   - What happens if we shrink the array size to say 256? To be expected, this is bad - way more instances of dropped characters.
415   -
416   - Added blink for each test to the OpenLog_Test sketch so we can see as the test progresses.
417   -
418   - http://www.sdcard.org/consumers/formatter/ is the site for SD card formatting. It looks like this program takes a guess at the correct block size. This could
419   - help a lot in the future.
420   -
421   -
422   - v2.4 Merged ringp updates. Commands cd, rm, ls work again! New "rm -rf" command to remove directory and its files.
423   -
424   - 29028 bytes of 30720 (yikes).
425   -
426   - Thanks ringp! Great work.
427   -
428   - Remember - we had to completely butcher HardwareSerial.cpp so a normal Arduino installation will not work.
429   - C:\arduino-xxxx\hardware\arduino\cores\arduino\HardwareSerial.cpp
430   - I've added the modified HardwareSerial.cpp to the
431   -
432   - Testing at 57600
433   - 1GB: 110490/111000, 110490/111000
434   - 8GB: 111000/111000, 111000/111000, 111000/111000
435   - 16GB: 83890/111000, 84908/111000
436   - The 16GB card with tons of files continue to have problems but the other cards (FAT and FAT32) are acceptable. Whenever possible, use a clean,
437   - empty, freshly formatted card.
438   -
439   - "rm -rf mydirectory" to remove a directory and all its files
440   -
441   - Windows 7 stores the Arduino hex file is an aweful place. Something like:
442   - C:\Users\Main\AppData\Local\Temp\build3390340147786739225.tmp\OpenLog_v2.cpp.hex
443   -
444   - Added HardwareSerial.cpp and a readme to the main trunk.
445   - Added OpenLog_v2.cpp.hex to the main trunk.
446   -
447   -
448   - v2.41 Power loss bug fixed. Adding support for 38400bps for testing with SparkFum 9DOF IMU logging.
449   -
450   - 29124 bytes of 30720 (yikes)
451   -
452   - Found a bug in the append_file routine. If the unit is actively logging, file.writes are flying by quickly. But if OpenLog loses power, none of
453   - the data is recorded because there is no file.sync. Data would only get recorded if the unit went idle or if user entered escape command. This
454   - has since been fixed with two file.sync() commands.
455   -
456   -
457   - v2.5 Improved 'read' command. Added 'reset' command.
458   -
459   - 28782 bytes of 30720 (yikes)
460   -
461   - Modified the read command so that we can print extended ASCII characters. The function was also failing to print any value over 127
462   - (limited to signed 8 bits - boo). Thank you wilafau! You proposed excellent, easy fixes.
463   -
464   - Added raw command during print operation. This allows for non-visible and extended ASCII characters to be printed correctly, if need be.
465   -
466   - Issuing 'reset' command causes OpenLog to reset and re-read the config file. This is important if you want to change the config file then restart.
467   -
468   - Removed some of the extraneous prints from the help menu to save on space.
469   -
470   -
471   - v2.51 Improved command prompt parsing (now ignores '\n')
472   -
473   - We now ignore the \n character when parsing characters from the command prompt. This makes it easier to create code on a microcontroller that
474   - correctly controls OpenLog. Previously, println or sprintf commands were adding a \n to the end of the string that would confuse OpenLog. One
475   - way around this if you have previous versions is this
476   - sprintf(buff, "new blah.txt\r");
477   - Serial.print(buff); //No println, use \r in string instead
478   -
479   - Also - added a CommandTest sketch to demonstrate how you can control OpenLog from a microcontroller / automate the command prompt
480   -
  1 + 12-3-09
  2 + SparkFun Electronics 2012
  3 + Nathan Seidle
  4 +
  5 + OpenLog hardware and firmware are released under the Creative Commons Share Alike v3.0 license.
  6 + http://creativecommons.org/licenses/by-sa/3.0/
  7 + Feel free to use, distribute, and sell varients of OpenLog. All we ask is that you include attribution of 'Based on OpenLog by SparkFun'.
  8 +
  9 + OpenLog is based on the work of Bill Greiman and sdfatlib: http://code.google.com/p/sdfatlib/
  10 +
  11 + OpenLog is a simple serial logger based on the ATmega328 running at 16MHz. The ATmega328
  12 + is able to talk to high capacity (larger than 2GB) SD cards. The whole purpose of this
  13 + logger was to create a logger that just powered up and worked. OpenLog ships with an Arduino/Optiboot
  14 + 115200bps serial bootloader running at 16MHz so you can load new firmware with a simple serial
  15 + connection.
  16 +
  17 + OpenLog automatically works with 512MB, 1GB, 2GB, 4GB, 8GB, and 16GB microSD cards. We recommend FAT16 for 2GB and smaller cards. We
  18 + recommend FAT32 for 4GB and larger cards.
  19 +
  20 + OpenLog runs at 9600bps by default. This is configurable to 2400, 4800, 9600, 19200, 38400, 57600, and 115200bps. You can alter
  21 + all settings including baud rate and escape characters by editing config.txt found on OpenLog.
  22 +
  23 + Type '?' to get a list of supported commands.
  24 +
  25 + During power up, you will see '12<'. '1' indicates the serial connection is established. '2' indicates
  26 + the SD card has been successfully initialized. '<' indicates OpenLog is ready to receive serial characters.
  27 +
  28 + Recording constant 115200bps datastreams are supported. Throw it everything you've got! To acheive this maximum record rate, please use the
  29 + SD card formatter from : http://www.sdcard.org/consumers/formatter/. The fewer files on the card, the faster OpenLog is able to begin logging.
  30 + 200 files is ok. 2GB worth of music and pictures is not.
  31 +
  32 + To a lower dir, use 'cd ..' instead of 'cd..'.
  33 +
  34 + Only standard 8.3 file names are supported. "12345678.123" is acceptable. "123456789.123" is not.
  35 +
  36 + All file names are pushed to upper case. "NewLog.txt" will become "NEWLOG.TXT".
  37 +
  38 + Type 'set' to enter baud rate configuration menu. Select the baud rate and press enter. You will then
  39 + see a message 'Going to 9600bps...' or some such message. You will need to power down OpenLog, change
  40 + your system UART settings to match the new OpenLog baud rate and then power OpenLog back up.
  41 +
  42 + If you get OpenLog stuck into an unknown baudrate, there is a safety mechanism built-in. Tie the RX pin
  43 + to ground and power up OpenLog. You should see the LEDs blink back and forth for 2 seconds, then blink
  44 + in unison. Now power down OpenLog and remove the RX/GND jumper. OpenLog is now reset to 9600bps.
  45 +
  46 + Please note: The preloaded Optiboot serial bootloader is 0.5k, and begins at 0x7E00 (32,256). If the code is
  47 + larger than 32,256 bytes, you will get verification errors during serial bootloading.
  48 +
  49 + STAT1 LED is sitting on PD5 (Arduino D5) - toggles when character is received
  50 + STAT2 LED is sitting on PB5 (Arduino D13) - toggles when SPI writes happen
  51 +
  52 + LED Flashing errors @ 2Hz:
  53 + No SD card - 3 blinks
  54 + Baud rate change (requires power cycle) - 4 blinks
  55 +
  56 + OpenLog regularly shuts down to conserve power. If after 0.5 seconds no characters are received, OpenLog will record any unsaved characters
  57 + and go to sleep. OpenLog will automatically wake up and continue logging the instant a new character is received.
  58 +
  59 + 1.55mA idle
  60 + 15mA actively writing
  61 +
  62 + Input voltage on VCC can be 3.3 to 12V. Input voltage on RX-I pin must not exceed 6V. Output voltage on
  63 + TX-O pin will not be greater than 3.3V. This may cause problems with some systems - for example if your
  64 + attached microcontroller requires 4V minimum for serial communication (this is rare).
  65 +
  66 + OpenLog has progressed significantly over the past three years. Please see Changes.txt or GitHub for a full change log.
  67 +
  68 +
  69 + v1.1
  70 + Adding better defines for EEPROM settings
  71 + Adding new log and sequential log functions.
  72 +
  73 + Code is acting very weird with what looks to be stack crashes. I can get around this by turning the optimizer off ('0').
  74 + Found an error : EEPROM functions fail when optimizer is set to '0' or '1'.
  75 + sd-reader_config.h contains flag for USE_DYNAMIC_MEMORY
  76 +
  77 + Looks like tweaking the optimization higher causes the asm("nop"); to fail inside the append_file routine. Changing this to
  78 + delay_us(1); works.
  79 +
  80 + I have a sneaking suspicion that I was having buffer overrun problems when defining the input_buffer at 1024 bytes. The
  81 + ATmega328 should have enough RAM (2K) but weird reset errors were occuring. With the buffer at 512bytes, append_file runs
  82 + just fine and is able to log at 115200 at a constant data rate.
  83 +
  84 + Added file called 'lots-o-text.txt' to version control. This text contains easy to scan text to be used for full data
  85 + rate checking and testing.
  86 +
  87 +
  88 + v1.2
  89 + ringp added:
  90 + Adding support for splitting command line parameters into arguments
  91 + Adding support for reading files sequencially
  92 + read <filename> <start> <length>
  93 + New log now for sequencial log functions supports 0-65535 files
  94 + Adding support for wildcard listing or deletion of files
  95 + ls <wildcard search>
  96 + rm <wildcard delete>
  97 +
  98 + Really great additions. Thanks ringp!
  99 +
  100 + Nate added error testing within newlog()
  101 + Checks to see if we have 65534 logs. If so, error out to command prompt with "!Too many logs:1"
  102 +
  103 +
  104 + v1.3
  105 + Added sd_raw_sync() inside append_file. I believe this was why tz1's addition of the timeout buffer update feature
  106 + was not working. Auto buffer update now working. So if you don't send anything to OpenLog for 5 seconds,
  107 + the buffer will automatically record/update.
  108 +
  109 + Need to create 'Testing' page to outline the battery of tests we need to throw at any OpenLog after a firmware
  110 + submission and update is complete.
  111 +
  112 + Testing
  113 + create 40,000 logs
  114 +
  115 + Record at full speed:
  116 + Run at full 115200, load lotsoftext.txt and verify no characters are dropped.
  117 +
  118 + Detect too many logs:
  119 + Create new log at 65533 (do this by editing 'zero' function to set EEPROM to 0xFF and oxFD)
  120 + and power cycle. Verify unit starts new log. Power cycle and verify unit errors out and drops to command prompt.
  121 +
  122 + Record buffer after timeout:
  123 + Create new log. Type 20 characters and wait 5 seconds. Unit should auto-record buffer. Power down unit.
  124 + Power up unit and verify LOG has 20 characters recorded.
  125 +
  126 +
  127 + v1.4
  128 + Added exit options to the two menus (set and baud)
  129 + Also added display of current settin to two menus (Ex: "Baud currently: 57600bps")
  130 +
  131 + Added '!' infront of 'error opening'. This pops up if there is an error while trying to append
  132 + to a freshly created new log (ex: LOG00548.txt is created, then errors out because it cannot append).
  133 + '!' was added so that user can parse against it.
  134 +
  135 + Replicated logging errors at 57600 using 5V Arduino
  136 + Unit would systematically glitch during logging of 111054 bytes
  137 +
  138 + Increasing buffer to 1000 characters caused URU error.
  139 + URU: Unit Resets Unexpectedly
  140 +
  141 + To recreate URU error. Type "append ". Include the space. If you get "!error opening", then things are
  142 + fine. If you get "!error opening#" where # is a weird character, then type 'ls' and the unit will
  143 + unexpectedly reset (URU error). I believe this is attributed to a memory overrun somewhere in the
  144 + FAT system.
  145 +
  146 + Changed buffer size to 900 and declared the character buffer as volatile
  147 + #define BUFF_LEN 900
  148 + volatile char input_buffer[BUFF_LEN];
  149 +
  150 + This increase to the buffer allows for clean logging of 444055 bytes with no URU errors.
  151 +
  152 + Experimenting with Scott's SD cards (customer gave cards on loan for recreating logging errors):
  153 + Card with single ~740mb file produces errors when trying to open/append to new log.
  154 + Card with less stuff on it logs full 444055 bytes correctly.
  155 +
  156 +
  157 + v1.5
  158 + Added 4800bps and 19200bps support
  159 +
  160 + Added power saving features. Current consumption at 5V is now:
  161 + In default append mode:
  162 + 6.6/5.5mA while receiving characters (LED On/Off)
  163 + 2.1mA during idle
  164 + In command mode: 3.2/2.1mA (LED On/Off)
  165 +
  166 + So if you're actively throwing characters at the logger, it will be ~6mA. If you send the logger
  167 + characters then delay 5-10 seconds, current will be ~2.5mA. (Unit records the characters in the buffer
  168 + and goes into idle more if no characters are received after 5 seconds)
  169 +
  170 + These power savings required some significant changes to uart.c / uart_getc()
  171 +
  172 +
  173 + v1.51 check_emergency_reset, default break character is ctrl+z 3 times, example Arduino sketch
  174 +
  175 + Added function from mungewell - check_emergency_reset. This has improved testing of the RX pin.
  176 + There was potential to get a false baud reset. There is still a chance but it's now much less likely.
  177 +
  178 + If OpenLog is attached to a Arduino, during bootloading of the Arduino, ctrl+z will most likely be sent
  179 + to the Arduino from the computer. This character will cause OpenLog to drop to command mode - probably
  180 + not what we want. So I added user selectable character (ctrl+x or '$' for example) and I added
  181 + user selectable number of escape characters to look for in a row (example is 1 or 2 or 3, '$$$' is a
  182 + common escape sequence). The default is now ctrl+z sent 3 times in a row.
  183 +
  184 + Added an example Arduino sketch (from ScottH) to GitHub so that people can easily see how to send characters to
  185 + OpenLog. Not much to it, but it does allow us to test large amounts of text thrown at OpenLog
  186 + at 57600bps.
  187 +
  188 +
  189 + v1.6 Adding config file.
  190 +
  191 + What happens if I reset the system by pulling RX low, but the config file has corrupt values in it?
  192 +
  193 + If config file has corrupt values in it, system will default to known values 9600/ctrl+z/3/newlog
  194 +
  195 + If config file is empty, system resets to known values
  196 +
  197 + After some massive testing, and lots of code to check for illegal states, it looks to be pretty stable.
  198 + The only problem is that we're running out of RAM. The buffer had to be decreased from 900 bytes
  199 + to 700 bytes to facilitate all the config file checking. Testing at 57600bps, unit runs very well
  200 + over 40kb test file on straight RS232 connection. That's pretty good. Testing at 115200 on straight
  201 + connection, unit will drop a buffer every once and a while. Not great, but not much we can do if the
  202 + SD card times out for ~150ms while it's writing.
  203 + 8 bits to the byte plus a start/stop bit = 10 bits per byte
  204 +
  205 + @ 9600bps = 960 bytes per second. Buffer will last for 729ms
  206 + @ 57600bps = 5760 bytes per second. Buffer will last for 121ms
  207 + @ 115200bps = 11520 bytes per second. Buffer will last for 60.7ms
  208 +
  209 + So if the SD card pauses for more than 60ms, 115200 will have lost data, sometimes. All other baud rates
  210 + should be covered for the most part.
  211 +
  212 + SD cards with larges amounts of data will have increased pause rates. Always use a clean card where possible.
  213 +
  214 +
  215 + v1.61 Small PCB change. Fixed version # in help menu.
  216 +
  217 + Fixed the firmware version in the help menu to v1.61.
  218 +
  219 + Updated Eagle files to Eagle v5.9. Fixed weird airwire. Changed D1 LED from Green to Blue.
  220 + Will only affect new production after 4-28-10.
  221 +
  222 + Closed some tickets and wrote some more example Arduino code:
  223 + http://forum.sparkfun.com/viewtopic.php?t=21438
  224 +
  225 +
  226 + v2.0 - 25986 bytes out of 30720
  227 + Welcome to version 2! We've moved from Roland Riegel's FAT library to Bill Greiman's sdfatlib. OpenLog now works with SD cards
  228 + up to 16GB (that is the current largest microSD card we can get our hands on). OpenLog automatically detects and works with FAT16/FAT32
  229 + file systems. It also automatically works with normal SD cards as well as SDHC cards.
  230 +
  231 + Almost all the previous commands have been ported from v1 to v2. The current commands that do not work:
  232 + cd.. - does not work. You can change to an upper directory, but you cannot navigate back down the tree.
  233 + cat - this command was depricated. HEX printing is now done with the 'read' command. We've added a 5th argument to select between ASCII and HEX printing.
  234 + Wild cards do not yet work. So rm and ls do not have wild cards enabled - yet. Help us out!
  235 +
  236 + Porting OpenLog to work directly under Arduino to work with sdfatlib (http://code.google.com/p/sdfatlib/) by Bill Greiman.
  237 +
  238 + sdfatlib intrinsically supports FAT16, FAT32 as well as SD and HCSD cards. In a word, it's amazing.
  239 +
  240 + Needs to be done:
  241 + Done - Get config file reading/loading working
  242 + Document config file in wiki: if no config file is found, current system settings are used. If config is found, system switches to settings found in file. If system settings are changed, then config file is changed and system uses new settings immediately.
  243 + Done - We don't want to constantly record a new config file on each power on. Only record when there is a change.
  244 + Get cd.. working
  245 + Seperate OpenLog_v2 into multiple files
  246 + Re-test too many log files created in the newlog setting - 65535. Potentially create thousands of files and see how sdfatlib handles it.
  247 + Done - Test sequential logging.
  248 + Get wild card support working for ls and rm
  249 + Get backspace working
  250 + Test rm with directories, and directory wildcards? Maybe not.
  251 + Get power save working
  252 + Test compile on a computer that doesn't have WinAVR
  253 +
  254 + Test commands:
  255 + new - works, also in sub dirs
  256 + append - works, also in sub dirs
  257 + rm - works, but does not yet support wild cards.
  258 + md - works, also in sub dirs
  259 + cd - you can change up to a sub-directory, but you cannot navigate back down the tree. The current work around is to type 'init'. This will re-init the card and set the directory back to root.
  260 + ls - works pretty well but does not yet support wild cards. Shows directories, files, and file sizes. Would be cool if it listed dirs first.
  261 + read - works well. Tested 0, 1, 2, 3, 4, and 5 arguments (included and left off). Fails gracefully. Now prints in HEX as well!
  262 + size - works well
  263 + disk - works well, prints more information than ever before!
  264 + init - works well
  265 + sync - works, but not really tested
  266 + cat - I've decided to drop this command. You can now print in hex using the read command and using a 5th argument as '1' for ASCII (default) and '2' for HEX.
  267 +
  268 + set - works well
  269 + baud - works well
  270 +
  271 +
  272 + v2.1 - Power save not working. Fixed issue 35. Dropping characters at 57600bps.
  273 + 26058 bytes out of 30720
  274 + Fixed a bug found by Salient (Issue 35). User settings where declared at chars which allowed them to be signed. If a user went from old firmware, to v2,
  275 + the safety checks would fail because the settings would be read at -1 instead of 255. Declaring user settings as byte fixed issue.
  276 +
  277 + Added "a) Clear user settings" to set menu. This allows us to completely wipe an OpenLog (user settings and config file) to see how it will respond
  278 + to future firmware changes.
  279 +
  280 + Improved the file 'size' command.
  281 +
  282 + Sequential logging is tested and works.
  283 +
  284 + Receive testing: Using the Test_Sketch found on Github, I am testing the receive reliability at different UART speeds.
  285 + We need to test a lot of received data. At 57600, 115200, and both from an Arduino (lots of time in between characters becuase of software overhead)
  286 + and from a raw serial port (almost no time in between characters). I am hoping to make sdfatlib hiccup at 115200, full speed, across a 1MB file. If
  287 + I can make it fail, then we can start to increase the buffer size and look at how much RAM sdfatlib has left open for the buffer.
  288 +
  289 + 9600bps from Arduino works fine
  290 + 57600bps from Arduino drops characters
  291 + 115200 from Arduino drops characters
  292 +
  293 + It seems that sdfatlib takes longer to write to the SD card than the original file system from Robert Reigel. I'm thinking perhaps
  294 + we should create a version of OpenLog firmware that is just sequantial logging, no fancy system menus... It might open up some RAM.
  295 +
  296 + If only we could get control of the UART from Arduino's clutches, we could probably handle the ring buffer much better. Not sure how to handle UART
  297 + interrupts without tearing out HardwareSerial.cpp.
  298 +
  299 + Added README to the Test sketch. Added 115200bps to test sketch.
  300 +
  301 +
  302 + v2.11 Tested with 16GB microSD. Fixed some general bugs. Lowered power consumption.
  303 +
  304 + 26136 bytes out of 30720
  305 +
  306 + Fixed issue 30. I added printing a period ('.') for non-visible ASCII characters during a 'read' command. This cleans up the output a bit. HEX
  307 + printing is still available.
  308 +
  309 + Fixed issue 34. When issuing a long command such as "read log00056.txt 100 200 2" (read from spot 100 to spot 200 and print in HEX), the
  310 + command shell would die at 24 spots. I increased both the general_buffer and 'buffer' in the command shell from 24 to 30. The limit is now
  311 + 30 characters, so "read log00056.txt 100 20000 2" is allowed.
  312 +
  313 + Works with a 16GB microSD card! High volume test: loaded 16GB card with 5GB of data. Basic serial tests work. When running at 57600, there
  314 + is an odd delay. I think it has to do with the file system doing an initial scan for an available cluster. Takes 2-3 seconds before OpenLog
  315 + returns to normal. This can cause significant data loss.
  316 +
  317 + Fixing power management in v2. Power down after no characters for 3 seconds now works. Unit drops to 2.35mA in sleep. 7.88mA in sitting
  318 + RX mode (awake but not receiving anything). There is still a weird bug where the unit comes on at 30mA. After sleep, it comes back at the
  319 + correct 7-8mA. Is something not getting shut off?
  320 +
  321 +
  322 + v2.2 Modified append_file() to use a single buffer. Increased HardwareSerial.cpp buffer to 512 bytes.
  323 +
  324 + More testing at 57600. Record times look to be 2, 5, and zero milliseconds when doing a record. This means that the split buffer doesn't
  325 + really make a difference? There are some records that take 150ms, 14ms, etc. At 57600bps, that's 7200 bytes/s, 138us per byte. With a 150ms
  326 + pause, that's 1,086 bytes that need to be buffered while we wait... Grrr. Too many.
  327 +
  328 + I re-wrote the append_file function to use only one buffer. This allows us to more quickly pull data from the hardware serial buffer. Hardware
  329 + serial buffer has to be increased manually to 512. This file (hardwareserial.cpp) is stored in the Arduino directory. With testing,
  330 + it seems like recording is working more solidly at 57600bps. But now I'm seeing glitches at 19200bps so more testing is required before we
  331 + make this the official OpenLog release.
  332 +
  333 + Moved input_buffer into within the append function. Attempting to shave bytes of RAM.
  334 +
  335 +
  336 + v2.21 ringp fork brought in. Wildcard remove and list commands now work. Remove directory now works! Change directory up/down the tree works again.
  337 +
  338 + 28440 bytes used of 30720.
  339 +
  340 + ringp brought back many great commands! Thanks ringp!
  341 + rm LOG*.* works
  342 + ls *.TXT works
  343 + cd .. works again
  344 + ls now correctly shows directories first and files following the directories.
  345 +
  346 + To remove a directory, you have to navigate into that directory. For example:
  347 + >cd TEMP (you are now in TEMP directory)
  348 + >rm -f TEMP (you are now removing TEMP, and you will be pushed back down one level of the tree)
  349 + >ls (shows files and directories where TEMP directory used to be, but TEMP directory should be gone)
  350 +
  351 + ringp added new commands:
  352 + efcount: gives the number of files in the current directory. Example: "efcount" returns "count|3". There are 3 files in the current directory.
  353 + efinfo <spot>: gives info about the file in <spot>. Example: "efinfo 2" reports "LOG00588.TXT|45". File number 2 is named LOG00588.TXT and is 45 bytes in size.
  354 + verbose <"on"|"off">: sets whether command errors are verbose (long winded) or just the "!" character. Example: "verbose off" sets verbose off. Then if a
  355 + command like "blah" is received, then only "!>" is seen from the command line interface. This makes it easier for embedded systems to recognize there
  356 + was an error. This setting is not recorded to EEPROM.
  357 +
  358 +
  359 + v2.3 Migrated to v10.10.10 of sdfatlib. Moved to inline RX interrupt and buffer. Improved the ability to receive a constant stream at 57600bps.
  360 +
  361 + 27334 bytes out of 30720.
  362 +
  363 + Remember - we had to completely butcher HardwareSerial.cpp so a normal Arduino installation will not work.
  364 + C:\arduino-xxxx\hardware\arduino\cores\arduino\HardwareSerial.cpp
  365 +
  366 + I removed the receive interupt handler from HardwareSerial.cpp so that we can deal directly with USART_RX_vect in the main code. This allows
  367 + us to return to the way OpenLog used to operate - by having one buffer split in half. Once one half fills, it is recorded to SD while the other
  368 + half fills up. This dramatically decreases the time spent in function calls and SD recording, which leads to many fewer characters dropped.
  369 +
  370 + The change log at the top of the main code got so large I've moved it to a version controlled "Changes.txt" file.
  371 +
  372 + By making all these changes, I have broken many things. Ringp - I could use your help here. I apologize for stomping on so much of your work. I was not
  373 + good enough to figure out how to re-link from the old function calls to the new sdfatlib setup.
  374 +
  375 + Backspace is broken - ringp, I saw this fix in one of your commits, but looking at the code, I don't see how it is supposed to work. Either way, we still
  376 + get 0x08 when trying to backspace.
  377 +
  378 + New sdfatlib doesn't have SdFat.cpp so fileInfo doesn't work. These function calls are marked with //Error
  379 +
  380 + I have chosen to dis-allow deprecated functions:
  381 + #define ALLOW_DEPRECATED_FUNCTIONS 0
  382 + This forced some trivial changes to the SD write and new file creation function calls. I believe we've successfully migrated to the new version of sdfatlib.
  383 +
  384 + In the command_shell / read_line function : It may be better to pull directly from the UART buffer and use the RX interrupt. For now, we brute force it.
  385 +
  386 + Because of all these changes, I need to re-test power consumption. For now, I'm confident it's low enough.
  387 +
  388 + Testing with 512 buffer array size
  389 + 1GB @ 57600 - dropped very little out of 3 tests
  390 + 1GB @ 115200 - dropped very little out of 2 tests
  391 + 8GB @ 57600 - Formatted using the sd formater (32k byte allocation size). Dropped nothing.
  392 + 8GB @ 115200 - dropped very little, dropped none
  393 + 16GB w/ Lots of files @ 57600 - Drops the first part of the file because of start up delay?
  394 + 16GB w/ Lots of files @ 115200
  395 +
  396 + 1024 array size (and 800) does not run
  397 +
  398 + Testing with 700 buffer array size
  399 + 1GB @ 57600 - 110300 out of 111000 bytes, 110300/111000,
  400 + 1GB @ 115200 - 111000/111000!, 109600/111000
  401 + 8GB @ 57600 - 109000/111000, 111000/111000!,
  402 + 8GB @ 115200 - 111000/111000!, 111000/111000!,
  403 + 16GB w/ Lots of files @ 57600 - 85120/111000, 85120/111000
  404 + 16GB w/ Lots of files @ 115200 - 56420 (but once it got going, log looks good). 56420.
  405 +
  406 + I am seeing long delays on cards with lots of files. In the above tests, the 16GB test card is a good example. It has 2GB worth of random files in a sub directory.
  407 + After OpenLog boots, goes to '12<'. After I send ~500 characters OpenLog freezes for a couple seconds, then returns to normal, very fast, operation. During
  408 + that down time, I believe sdfatlib is searching for an open cluster. The odd thing is that after the cluster is established (after the initial down time) OpenLog
  409 + performs excellently. I am hoping to create a faux file or pre-write and save to the file or some how get this allocation done before we report the
  410 + '12<' indicating we are ready. That way, if a card does fill up, as long as the host system is checking for '<', it doesn't matter how long it takes
  411 + sdfatlib to find the next free cluster.
  412 +
  413 + You can see that we drop 700 bytes at a time. That's a bit odd - I'd expect to drop half or 350 at a time.
  414 + What happens if we shrink the array size to say 256? To be expected, this is bad - way more instances of dropped characters.
  415 +
  416 + Added blink for each test to the OpenLog_Test sketch so we can see as the test progresses.
  417 +
  418 + http://www.sdcard.org/consumers/formatter/ is the site for SD card formatting. It looks like this program takes a guess at the correct block size. This could
  419 + help a lot in the future.
  420 +
  421 +
  422 + v2.4 Merged ringp updates. Commands cd, rm, ls work again! New "rm -rf" command to remove directory and its files.
  423 +
  424 + 29028 bytes of 30720 (yikes).
  425 +
  426 + Thanks ringp! Great work.
  427 +
  428 + Remember - we had to completely butcher HardwareSerial.cpp so a normal Arduino installation will not work.
  429 + C:\arduino-xxxx\hardware\arduino\cores\arduino\HardwareSerial.cpp
  430 + I've added the modified HardwareSerial.cpp to the
  431 +
  432 + Testing at 57600
  433 + 1GB: 110490/111000, 110490/111000
  434 + 8GB: 111000/111000, 111000/111000, 111000/111000
  435 + 16GB: 83890/111000, 84908/111000
  436 + The 16GB card with tons of files continue to have problems but the other cards (FAT and FAT32) are acceptable. Whenever possible, use a clean,
  437 + empty, freshly formatted card.
  438 +
  439 + "rm -rf mydirectory" to remove a directory and all its files
  440 +
  441 + Windows 7 stores the Arduino hex file is an aweful place. Something like:
  442 + C:\Users\Main\AppData\Local\Temp\build3390340147786739225.tmp\OpenLog_v2.cpp.hex
  443 +
  444 + Added HardwareSerial.cpp and a readme to the main trunk.
  445 + Added OpenLog_v2.cpp.hex to the main trunk.
  446 +
  447 +
  448 + v2.41 Power loss bug fixed. Adding support for 38400bps for testing with SparkFum 9DOF IMU logging.
  449 +
  450 + 29124 bytes of 30720 (yikes)
  451 +
  452 + Found a bug in the append_file routine. If the unit is actively logging, file.writes are flying by quickly. But if OpenLog loses power, none of
  453 + the data is recorded because there is no file.sync. Data would only get recorded if the unit went idle or if user entered escape command. This
  454 + has since been fixed with two file.sync() commands.
  455 +
  456 +
  457 + v2.5 Improved 'read' command. Added 'reset' command.
  458 +
  459 + 28782 bytes of 30720 (yikes)
  460 +
  461 + Modified the read command so that we can print extended ASCII characters. The function was also failing to print any value over 127
  462 + (limited to signed 8 bits - boo). Thank you wilafau! You proposed excellent, easy fixes.
  463 +
  464 + Added raw command during print operation. This allows for non-visible and extended ASCII characters to be printed correctly, if need be.
  465 +
  466 + Issuing 'reset' command causes OpenLog to reset and re-read the config file. This is important if you want to change the config file then restart.
  467 +
  468 + Removed some of the extraneous prints from the help menu to save on space.
  469 +
  470 +
  471 + v2.51 Improved command prompt parsing (now ignores '\n')
  472 +
  473 + We now ignore the \n character when parsing characters from the command prompt. This makes it easier to create code on a microcontroller that
  474 + correctly controls OpenLog. Previously, println or sprintf commands were adding a \n to the end of the string that would confuse OpenLog. One
  475 + way around this if you have previous versions is this
  476 + sprintf(buff, "new blah.txt\r");
  477 + Serial.print(buff); //No println, use \r in string instead
  478 +
  479 + Also - added a CommandTest sketch to demonstrate how you can control OpenLog from a microcontroller / automate the command prompt
  480 +
481 481
0  OpenLog_v3/OPENLOG_V311_Combined_3_30_12.hex → ...ware/OpenLog_v3/OPENLOG_V311_Combined_3_30_12.hex
File renamed without changes
3,616 OpenLog_v3/OpenLog_v3.cpp.hex → firmware/OpenLog_v3/OpenLog_v3.cpp.hex
1,808 additions, 1,808 deletions not shown
0  OpenLog_v3/OpenLog_v3.ino → firmware/OpenLog_v3/OpenLog_v3.ino
File renamed without changes
2,198 OpenLog_v3_Light/OpenLog_v3_Light.cpp.hex → firmware/OpenLog_v3_Light/OpenLog_v3_Light.cpp.hex
... ... @@ -1,1099 +1,1099 @@
1   -:100000000C94CC000C94F4000C94F4000C94F400C8
2   -:100010000C94F4000C94F4000C94F4000C94F40090
3   -:100020000C94F4000C94F4000C94F4000C94F40080
4   -:100030000C94F4000C94F4000C94F4000C94F40070
5   -:100040000C94E61D0C94F4000C94021D0C94481DB5
6   -:100050000C94F4000C94F4000C94F4000C94F40050
7   -:100060000C94F4000C94F4006572726F7220777235
8   -:100070006974696E6720746F2066696C65004372ED
9   -:100080006561746520636F6E666967206661696C7F
10   -:1000900065640052656D6F766520636F6E66696793
11   -:1000A000206661696C6564004572726F72206372CC
12   -:1000B000656174696E67205345514C4F470A0021B2
13   -:1000C000546F6F206D616E79206C6F67733A3221C7
14   -:1000D0000021546F6F206D616E79206C6F67733AE9
15   -:1000E0003121006572726F72206F70656E526F6F92
16   -:1000F00074006572726F7220766F6C756D652E6913
17   -:100100006E6974006572726F7220636172642E6929
18   -:100110006E6974005344206572726F723A200065F4
19   -:1001200072726F723A20007C3C3E5E2B3D3F2F5B2B
20   -:100130005D3B2C2A225C0000000000240027002ADE
21   -:100140000000000000250028002B00000000002314
22   -:10015000002600290004040404040404040202022A
23   -:1001600002020203030303030301020408102040F8
24   -:100170008001020408102001020408102000000081
25   -:100180000700020100000304060000000000000058
26   -:100190000000E404BA17DD0411241FBECFEFD8E03D
27   -:1001A000DEBFCDBF11E0A0E0B1E0E2EBF3E402C0BE
28   -:1001B00005900D92A63DB107D9F717E0A6EDB1E085
29   -:1001C00001C01D92A739B107E1F711E0C6E9D1E0FE
30   -:1001D00004C02297FE010E949121C239D107C9F7BC
31   -:1001E0000E94B61F0C94CC210C940000FC018827BF
32   -:1001F0009927E89421912032E9F3293010F02E302C
33   -:10020000C8F32B3241F02D3239F4689404C00E94B7
34   -:100210004501820F911D219120532A30C0F31EF415
35   -:10022000909581959F4F0895FB01DC0104C08D914D
36   -:100230000190801921F441505040C8F7881B990B58
37   -:100240000895FB01DC0102C001900D924150504025
38   -:10025000D8F70895FB01DC010D900020E9F7119714
39   -:1002600001900D920020E1F70895FB01DC018D91D2
40   -:10027000019080190110D9F3990B0895FB01DC015D
41   -:1002800001900D920020E1F708957AE0979F902D5C
42   -:10029000879F802D910D11240895AEE0B0E0E3E535
43   -:1002A000F1E00C94A3210D891E8986E08C831A83CA
44   -:1002B00009838FEF9FE79E838D839E01275E3F4FCB
45   -:1002C000CE0101966F89788DA9010E9471012F815D
46   -:1002D0003885020F131FF80110822E96E4E00C946B
47   -:1002E000BF21ABE0B0E0E7E7F1E00C9495213C01E1
48   -:1002F0002B015A01FC0117821682838181FD03C004
49   -:100300006FEF7FEFC6C19AE0892E1E010894211C71
50   -:10031000311CF3012381F20123FD859123FF81919B
51   -:100320002F01882309F4B2C1853239F423FD859168
52   -:1003300023FF81912F01853229F490E0B3010E94BF
53   -:100340006403E7CF982FFF24EE249924FFE1FF15E3
54   -:10035000D0F09B3269F09C3228F4903259F09332FD
55   -:1003600091F40EC09D3249F0903369F441E024C00D
56   -:1003700052E0F52A84E0F82A28C098E0F92A25C03E
57   -:10038000E0E1FE2A22C0F7FC29C0892F80538A3081
58   -:1003900070F4F6FE05C0989C902C1124980E15C0A0
59   -:1003A000E89CE02C1124E80EF0E2FF2A0EC09E32F9
60   -:1003B00029F4F6FC6BC140E4F42A07C09C3619F41A
61   -:1003C00050E8F52A02C0983649F4F20123FD9591D0
62   -:1003D00023FF91912F01992309F0B8CF892F8554DC
63   -:1003E000833018F08052833038F444E050E0A40E9B
64   -:1003F000B51E5FE359830FC0933631F0933779F020
65   -:10040000933509F056C020C0F5018081898342E010
66   -:1004100050E0A40EB51E610101E010E012C0F5012C
67   -:10042000C080D180F6FC03C06FEF7FEF02C0692D62
68   -:1004300070E042E050E0A40EB51EC6010E945903D0
69   -:100440008C015FE7F52214C0F501C080D180F6FC75
70   -:1004500003C06FEF7FEF02C0692D70E042E050E013
71   -:10046000A40EB51EC6010E944E038C0150E8F52A69
72   -:10047000F3FE07C01AC080E290E0B3010E9464035B
73   -:10048000EA948E2D90E008171907A8F30EC0F60124
74   -:10049000F7FC8591F7FE81916F0190E0B3010E9416
75   -:1004A0006403E110EA94015010400115110579F739
76   -:1004B000EAC0943611F0993669F5F7FE08C0F501E7
77   -:1004C000208131814281538184E090E00AC0F501AE
78   -:1004D000808191819C01442737FD4095542F82E013
79   -:1004E00090E0A80EB91E9FE6F92257FF09C050956B
80   -:1004F0004095309521953F4F4F4F5F4FE0E8FE2AE2
81   -:10050000CA01B901A1012AE030E00E949003D82E6F
82   -:10051000D21840C0953729F41F2D1F7E2AE030E005
83   -:100520001DC01F2D197F9F3661F0903720F498353C
84   -:1005300009F0ACC00FC0903739F0983709F0A6C069
85   -:1005400004C028E030E00AC0106114FD146020E10E
86   -:1005500030E004C014FD166020E132E017FF08C04F
87   -:10056000F501608171818281938144E050E008C08F
88   -:10057000F50180819181BC0180E090E042E050E093
89   -:10058000A40EB51EA1010E949003D82ED2188FE7A9
90   -:10059000F82EF122F6FE0BC05EEFF522D91438F4E6
91   -:1005A000F4FE07C0F2FC05C08FEEF82202C01D2D3C
92   -:1005B00001C0192DF4FE0DC0FE01ED0DF11D80816D
93   -:1005C000803319F499EEF92208C01F5FF2FE05C0CE
94   -:1005D00003C08F2D867809F01F5F0F2DF3FC14C028
95   -:1005E000F0FE0FC01E1510F09D2C0BC09D2C9E0C14
96   -:1005F000911A1E2D06C080E290E0B3010E946403B0
97   -:100600001F5F1E15C0F304C01E1510F4E11A01C0CF
98   -:10061000EE2404FF0FC080E390E0B3010E94640366
99   -:1006200002FF1DC001FD03C088E790E00EC088E511
100   -:1006300090E00BC0802F867891F001FF02C08BE222
101   -:1006400001C080E2F7FC8DE290E0B3010E946403F8
102   -:1006500006C080E390E0B3010E9464039A94D91429
103   -:10066000C0F3DA94F101ED0DF11D808190E0B3014A
104   -:100670000E946403DD20A9F706C080E290E0B30188
105   -:100680000E946403EA94EE20C1F743CEF301668131
106   -:100690007781CB012B96E2E10C94B121FC0105900E
107   -:1006A000615070400110D8F7809590958E0F9F1F74
108   -:1006B0000895FC016150704001900110D8F78095B9
109   -:1006C00090958E0F9F1F08950F931F93CF93DF93E5
110   -:1006D0008C01EB018B8181FF1BC082FF0DC02E813D
111   -:1006E0003F818C819D812817390764F4E881F98165
112   -:1006F0000193F983E88306C0E885F985802F099581
113   -:10070000892B31F48E819F8101969F838E8302C055
114   -:100710000FEF1FEFC801DF91CF911F910F91089547
115   -:10072000FA01AA27283051F1203181F1E8946F9322
116   -:100730006E7F6E5F7F4F8F4F9F4FAF4FB1E03ED0C8
117   -:10074000B4E03CD0670F781F891F9A1FA11D680F66
118   -:10075000791F8A1F911DA11D6A0F711D811D911D99
119   -:10076000A11D20D009F468943F912AE0269F11240E
120   -:100770003019305D3193DEF6CF010895462F477072
121   -:10078000405D4193B3E00FD0C9F7F6CF462F4F70CD
122   -:10079000405D4A3318F0495D31FD4052419302D02B
123   -:1007A000A9F7EACFB4E0A6959795879577956795D1
124   -:1007B000BA95C9F700976105710508959B01AC01D1
125   -:1007C0000A2E06945795479537952795BA95C9F7F8
126   -:1007D000620F731F841F951FA01D0895EF92FF9253
127   -:1007E0000F931F937A018B011092C1002E732093F7
128   -:1007F000C200403081EE580780E0680780E078074B
129   -:1008000029F41092C00020E130E013C082E0809310
130   -:10081000C00060E079E08DE390E0A80197010E94BC
131   -:100820006D212150304040405040569547953795B6
132   -:1008300027953093C5002093C40088E98093C100B8
133   -:100840001F910F91FF90EF9008958091C00085FF58
134   -:10085000FCCF6093C60081E090E00895CF92DF92D4
135   -:10086000EF92FF920F931F93CF93DF937C016B0165
136   -:100870008A01C0E0D0E00DC0D701ED91FC91D60116
137   -:10088000AC0FBD1F0190F081E02DC7016C9109955F
138   -:100890002196C017D10780F3C801DF91CF911F9136
139   -:1008A0000F91FF90EF90DF90CF90089508958DE71E
140   -:1008B00097E00E94EB1C0895DF93CF930F92CDB782
141   -:1008C000DEB78DE797E0BE016F5F7F4F0E94B41CDB
142   -:1008D000882319F42FEF3FEF03C08981282F30E0E0
143   -:1008E000C9010F90CF91DF9108958DE797E00E94A5
144   -:1008F000A21C0895FC0182ED91E0918380838781A1
145   -:10090000882321F0CF0104960E947B100895813046
146   -:10091000910509F044C06F5F7F4F09F04DC0109200
147   -:10092000D9011092D80188EE93E0A0E0B0E0809366
148   -:10093000DA019093DB01A093DC01B093DD018AEB37
149   -:1009400091E09093D7018093D6018DE797E062EE16
150   -:1009500071E041E253E00E94DF1C89E180930505CC
151   -:10096000109208051092200510922B0510922E056A
152   -:1009700010922A051092290582ED91E090932805A6
153   -:100980008093270510924A0510924D051092490553
154   -:100990001092480590934705809346050895892B4A
155   -:1009A00059F46F5F7F4F41F486E495E00E947A042A
156   -:1009B00087E295E00E947A04089580E090E06FEF6E
157   -:1009C0007FEF0E948704089581E090E06FEF7FEF52
158   -:1009D0000E94870408951F938DE897E061E070E01E
159   -:1009E0000E94D31D809365058B3050F081E0809389
160   -:1009F00065058DE897E061E070E041E00E94DA1D56
161   -:100A00008DE897E062E070E00E94D31D8093660558
162