Show progress for downloading remote files

[KyleAMathews]

Many plugins rely on createRemoteFileNode from gatsby-source-filesystem to download files from remote systems. E.g. gatsby-source-wordpress downloads all the files from the wordpress instance so you can include them in your site and do data transformations e.g. image thumbnail creation.

This is great but some systems have many files which can take a long time to download. This downloading can cause confusion because the Gatsby bootstrap just stalls out for no apparent reason.

To start improving this, the first thing we need to do is to show a progress bar & current status for downloads.

Related issue about problems this causes with wordpress #6654

[polarathene]

Also related: #6125

Over 11 minutes on source and transform nodes for me with one of the example projects(gatsby-image that pulls 130MB remote images from unsplash). Before being aware of the large remote content being downloaded I thought the build process had become stuck.

My suggestion in the linked issue was to consider a (sourced_nodes/total_nodes) feedback counter for the step with a timer delay before showing some hints to why the process might be taking long to the developer(especially useful and perhaps easier to implement if knowing the total nodes/remote content in advance is not an easy feature to provide). Linked issue contains some example messages/tips that could cycle.

I have also noted in that example project, that when an image was sourced remotely(at least in that project/unsplash case), if the transfer fails during, no error was raised, instead the build is considered successful and the images that failed will display their partially downloaded contents. Should I raise another issue about that? I'm not quite sure how I could setup a reproducible project to demonstrate or if Gatsby would have anyway to confirm the remote download was successful?

[oorestisime]

I would love to give this a go since i use that api quite often. I am assigning it to myself to avoid double work.