FTP plugin

[vooon]

I want to add FTP plugin when it support will stabilize.

Wait when:

• It will use FILE_TRANSFER_PROTOCOL
• PX4 will support all file operations FTP: write support PX4/PX4-Autopilot#1363

Also i need reference client implementation, so wait QGC update.

Note: we start discussion in #41, but FTP is offtopic there.

Implement file operations:

• list directory
• open file (read)
• read file
• close file
• create file (write)
• open file (write)
• write file
• mkdir
• rmdir
• remove file
• truncate file
• rename
• get crc32 for file

TODO:

• ACK timeout
• Return error code from open/read/write
• Bitrate flow stats
• Diagnostics
• Translate protocol errors to errno

Protocol TODO:

• open for write
• calculate crc32 for file (might be useful for verification)
• truncate()
• rename file/dir

[DonLakeFlyer]

I've been bogged down by finishing up a big QGroundControl RC Calibration pull. That went in yesterday. Now I move back to FTP. First work item will be improving download speed, with that work I"ll likely switch to the new mavlink message. Implementing missing file operations will come last and may take a while to get to. Hoping for first work item complete in a week. It may take longer since my concept for streaming messages to client without Acks may or may not work.

[TSC21]

Already uses FILE_TRANSFER_PROTOCOL: PX4/PX4-Autopilot@2c72468

[vooon]

I've started to write plugin.

@DonLakeFlyer i have few questions:

1. Why ftp adds extra crc32, while transfer layer (mavlink) already implements crc16 for datagrams?
2. There is no commands for mkdir()/rmdir()?
3. How i can get file stats?

[DonLakeFlyer]

1. Hmm, Never thought about that. Just following the original implementation. I"ll ask Lorenz at some point.
   2/3. Original driver to FTP was mainly to download logs. So not much work/thought put into 2 and 3.

[DonLakeFlyer]

Talked with Lorenz. Inner CRC is not needed. I'll remove it at some point. Once I get past download streaming I'll also implement full create/delete on files and directories. Commands for that are not fully in there yet.

[vooon]

_3. I've see in FTP, size transmitted as text in dir list, so at least file size can be cached in fuse driver.

[vooon]

But i don't see reasons why dir list return absolute path, not relative as readdir() do it.

[DonLakeFlyer]

List command returned paths are relative

[vooon]

I messed up a code piece which is taken stat().

[vooon]

Also i don't like that client don't know what command return ACK/NACK (from message).

[vooon]

FTP:List works!

TODO: timeout handling.

$ rosservice call /mavros/ftp/list /fs/microsd
list: 
  - 
    name: bootlog.txt
    type: 0
    size: 543
  - 
    name: dataman
    type: 0
    size: 101504
  - 
    name: log
    type: 1
    size: 0
  - 
    name: etc
    type: 1
    size: 0
success: True

[vooon]

Now it works (list & read)!

New script mavftp.

[TSC21]

@vooon if I want to download a file what's the procedure?

[vooon]

@TSC21 Start mavros, then use mavftp tool to list files:

rosrun mavros mavftp list /
rosrun mavros mavftp cat /etc/init.d/rc.usb > /tmp/file

Upd: FW has session leak bug, so after each 2 cat to one reset.

[vooon]

Check file size using list. But i not recommend use plugin for large files (>10K) now.
Also you can use pv to display data rate.

rosrun mavros mavftp cat /file | pv -rb -s <file size> > /tmp/file

[TSC21]

Check file size using list. But i not recommend use plugin for large files (>10K) now.

It took some time but I was able to download a 2.9MB file :) so it seems to be working. It is slow though.

Also you can use pv to display data rate.

rosrun mavros mavftp cat /file | pv -rb -s <file size> > /tmp/file

Can you include | pv -rb -s <file size> as default in the mavftp cat command? Or at least as a arg option like -r or --rate?

[vooon]

Maybe later, but mavftp goal is testing ftp plugin, not for general use.

Also cat require mavros read all file content and then return it via service, not fast on big files.

$ rosrun mavros mavftp cat /fs/microsd/log/2014-08-26/15_37_25.bin | pv -rab -s 298820 > /tmp/file.bin
 292kB [  24kB/s] [  24kB/s]

If pv is not wrong with the calculation speed, it is not that bad, with QGC like speed was 8 kB/s.

[DonLakeFlyer]

See: PX4/PX4-Autopilot#1345 for significant updates to FTP firmware. You'll have the update to the new opcodes and error codes.

[vooon]

Waiting PX4/PX4-Autopilot#1357 . Also maybe better to add file permissions on list operation.

[vooon]

Found error in kCmdListDirectory.

[vooon]

I'm done. Diagnostics will be later in #154.

Now i only waits merge PX4/PX4-Autopilot#1363.

[vooon]

I think i can close issue now, so as not to slow down release.