Multi packets http pages with the arduino

[fredycpu]

i have modified your library for sending webpages with bigger length as a frame

EDIT :WORKING CODE ON 15 july 2013!!!!!
I figured out compatibility problem between SD library in arduino ide and ethercard lib
switched to tinyFAT library and works very well
transfered 240 Megabytes from SD card wihout any packet loss !
Transfer speed :130 kilobytes /second

what i have changed :(this is the final code with bug corrections)

// added in ethercard.h :

static void httpServerReply_with_flags (uint16_t dlen , byte flags);
static void httpServerReplyAck ();

// added in tcpiip.cpp (declarations)

static unsigned long SEQ;

// added in tcpiip.cpp (functions)

static void get_seq() { //get the sequence number of packets after an ack from GET
SEQ =(((unsigned long)gPB[TCP_SEQ_H_P]_256+gPB[TCP_SEQ_H_P+1])_256+gPB[TCP_SEQ_H_P+2])*256+gPB[TCP_SEQ_H_P+3];
} //thanks to mstuetz for the missing (unsigned long)

static void set_seq(word dlen) { //set the correct sequence number and calculate the next with the lenght of current packet
gPB[TCP_SEQ_H_P]= (SEQ & 0xff000000 ) >> 24;
gPB[TCP_SEQ_H_P+1]= (SEQ & 0xff0000 ) >> 16;
gPB[TCP_SEQ_H_P+2]= (SEQ & 0xff00 ) >> 8;
gPB[TCP_SEQ_H_P+3]= (SEQ & 0xff );
SEQ=SEQ+dlen;//next sequence number
}

void EtherCard::httpServerReplyAck () {
make_tcp_ack_from_any(info_data_len,0); // send ack for http get
get_seq(); //get the sequence number of packets after an ack from GET
}

void EtherCard::httpServerReply_with_flags (word dlen , byte flags) {
set_seq(dlen);
gPB[TCP_FLAGS_P] = flags; // final packet
make_tcp_ack_with_data_noflags(dlen); // send data
}

Here you will find the actual version of this libray with my updates and
two working exemples (sending http pages in more packets from programm, and sending http content from sd card)

the multipacketSD exemple uses about 15 kb of flash but dont forget to fill sd card with exemple content /library/exemples/multipackesSD/copy to sd !!! headers files are needed (.hea)

http://microinfo.fr/multipackets.rar

Sorry for my english , i'm french
Frédéric HERMS

[fredycpu]

this the code i have used with two exemples :
the first send a page with 5 packets
the second, send files from sd card
with this exemple http://192.168.0.66/sd/picture.jpg will show you
picture.jpg on the sd card but with this code , the arduino need headers
files located on the sd card too. you can make headers files for any
types of files txt html jpg gif png .............. it's a facility for
unload the progmem of arduino.
aduino can send small files about 100kb with this method but for bigger
files, it will hangs i think the receive buffer is full and one packet
will not sended.
i hope it can help anyone.
if you have questions ask me
i hope my english is not too bad and you can understand me correctly.

i wish you a good day
fred.

[fabiopulzi]

Hello Fred, I'm just stating with Arduino development, and as I understood your hack is exactly what I need on my project, I made all the library changes, but unfortunately I couldn't make it work with my knowledge, could please post a simple usage example ?

Thanks in advance,
Fabio Pulzi

[fredycpu]

hello i send you the used code the exemple multipacketsd contain files
to copied on the sd card
try this code it's working on an arduino mega with mega ethernet shield

i wish you a good day

fred

[fabiopulzi]

Hello Fred, Thanks for your answer, did you send the code? Unfortunately I haven't receive it, could you send to mais email ? fabiopulzi at gmail.com

Thanks and have a nice day !

[fabiopulzi]

Hello Fred, I think that when you just answer the email with the file attached is not working, because it's an email from github and the file is not been attached there. Could you answer on github page and attached the file there ? Thanks in advance

[mzartmann]

Hi, i need for a webserver a transmit and upload routine for transfer large files ~300k-500k to and from arduino.
How can i make this???? the library is well undocumented ;-(
Or is anywhere antoher library ?

[konfiot]

@mzartmann : First, do you know where you will store this data, SD card ? and as far as I know file upload isn't implemented yet, even with the official library (because arduino isn't really done for these kind of applications)
and third thing, if you want more answers, i think you should create a new issue ...

[mzartmann]

@konfiot thanks for your reply!
yes i store the data on a SD Card, the arduino mega is in my case a datalogger with webserver.
I need to Download the logged Data and upload to maintain the webserver sites.
Now i will test the EtherBright library thats based on avr-uip, the documentation is promising (I hope i gets running :-) )

[vooc]

Great job Fred!!
your code modification very helpful.

i get same error when i try to serving more than 100k files.
i have webpage, that include some js files and css files.
after a while, browser cannot receive any packet anymore, although arduino gets the request, and send the whole file. using wireshark, i can see the packet arrived at eth0 with lots of retransmission error (i think the ack and seq number is not right. for example seq: 4294901864, and the previous packet has seq number 1).

my arduino ip is 192.168.3.124

You said that this problem may be from the receive buffer, is it the buffer inside enc28j60 chip? or the Ethernet::buffer?

Thank you

*again, great job.

[mstuetz]

Hi!
We ran sporadically into the same problem. Our problem was, that the sometimes the sequence number did not match the previous packet.

After some code reviewing we found the bug:
In your canges in the library you choose a too small variable for calculating the sequence number:

Simply replace:
SEQ =((gPB[TCP_SEQ_H_P]_256+gPB[TCP_SEQ_H_P+1])_256+gPB[TCP_SEQ_H_P+2])*256+gPB[TCP_SEQ_H_P+3];

with:
SEQ =(((unsigned long)gPB[TCP_SEQ_H_P]_256+gPB[TCP_SEQ_H_P+1])_256+gPB[TCP_SEQ_H_P+2])*256+gPB[TCP_SEQ_H_P+3];

After this change our little server runs very fine.

BTW: We found also out, that sending many small packets is much faster than sending less big ones.

[fredycpu]

mstuetz you do a great job! I had verified the capacity of variable type but i have missed something i think
thank you.
after verifications i dont really understand my error at the start of tcpip.cpp i declare SEQ as unsigned long.

the next step is to upload something to the arduino ;-)

[mstuetz]

Pls. see http://www.eskimo.com/~scs/cclass/int/sx4bb.html

Can someone take over this changes to the github source?

[vicatcu]

@mstuetz in reference to your last comment, what do you want to have done and why?

[fredycpu]

My work is finsh,

if you want ; you can take over this changes to the github source.

if you have questions i will try to help you.

[vicatcu]

I like this and will see what I can do to help create the patch and pull request. It will probably not be until next week though.Where is the latest source, just in this thread?

[fredycpu]

Le 15/07/2013 18:27, Victor Aprea a écrit :

I like this and will see what I can do to help create the patch and
pull request. It will probably not be until next week though.Where is
the latest source, just in this thread?

—
Reply to this email directly or view it on GitHub
#68 (comment).

at the beginning of the threat is the latest version of ethercard with
my mods 2 exemples and the tinySD library to make working the sd card
exemple

i have edited the issue openning message there are the latest news
about this.

[renatoaloi]

fredycpu, thank you for your EtherCard library enhancement! I am trying to accomplish that for 2 years now. Thanks Google i have found your work! I have developed a Lib to compile W5100 source-code into ENC28J60 hardware. Sorry about my english, too. I am brazilian. Take a look at: github.com/renatoaloi -- I would like to learn what you think about my work... There are 2 versions. Enc28CoreLib, about compiling W5100 code to ENC28J60 hardware; and EtherEncLib, a standalone multipacket implementation for ENC28J60. Do you send me some feedback about your thoughts? Thanks, Renato

[vicatcu]

@renatoaloi your libraries sound really cool, I'll have to give them a try on Nanode!

[renatoaloi]

Thanks for your reply, Victor! Keep in mind it's a newbie's work :)

On Tue, Dec 3, 2013 at 7:25 PM, Victor Aprea notifications@github.comwrote:

@renatoaloi https://github.com/renatoaloi your libraries sound really
cool, I'll have to give them a try on Nanode!

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/68#issuecomment-29753005
.

INTI Comercio de Componentes Eletronicos MEI
(11) 2592-6530
(11) 98134-8682
Rua Jose Lopes da Silva, 263 - SP,SP

[solarkennedy]

@jcw this should be closable since #93 was merged.