[Question] Best practice for executing a sequence of HTTP requests

[kiwiswift]

I'm using ProcedureKit in my project for downloading data from an API and I'm not sure if what I'm doing is a good (and safe) approach.

Basically I have an array of strings representing profile ids that I need to fetch from an endpoint. I'm creating an array of Procedures inside a GroupProcedure, and I'm adding an observer for each one to collect the fetched data and put in an array of profiles. When the GroupProcedure finishes, I move this array to the output property.

Here is my code:

        class ProfilesProcedure: GroupProcedure, InputProcedure, OutputProcedure {
            
            typealias T = [Profile]
            
            var input: Pending<[String]> = .pending
            
            public var output: Pending<ProcedureResult<T>> = .pending
            
            var profiles = T()
            
            init(profileIds: [String]) {
                
                //Create an array of URLrequests
                let requests = profileIds.compactMap { ProfileAPI.(id: $0).request() }
                
                //Create an array of Procedures
                let procedures = requests.compactMap { NetworkProcedure(request: $0) }
                
                super.init(operations: procedures)
                
                //Add an observer to each Procedure
                procedures.forEach { procedure in
                    procedure.addDidFinishBlockObserver { [weak self] procedure, error in
                        if let value = procedure.output.success, let profile = value {
                            self?.profiles.append(profile)
                        } else {
                            self?.finish(with: error)
                        }
                    }
                }
            }
            
            //When finished, copy array to output
            override func procedureDidFinish(with: Error?) {
                self.output = .ready(.success(self.profiles))
            }
            
        }

I wonder if this logic makes sense or if there's any better way to achieve this, please?

[danthorpe]

Hi @kiwiswift - sorry, only just spotted this question - will get back to you today. It's a good great question.

[danthorpe]

Okay, so first of all - you're definitely on the correct path, and more or less what you've got is exactly right. However because this is such a common thing to need to do, ProcedureKit has some convenience procedures to help you do this.

So, I'll summarize your problem (to help anyone else learning from this), and show you a slightly different way of doing this.

The problem

1. We have an array of Profile IDs:
2. Then construct a URL request from each Profile ID
3. Then execute N requests
4. Then parse the N responses into Profile objects

So, ultimately we've got a single task, which we want to do many times, and collect the results at the end.

Create a procedure for a single item

Therefore, start off by creating a Procedure which will receive a String (the profile ID) and return a Profile. It might look something like this:

final class GetProfileProcedure: GroupProcedure, InputProcedure, OutputProcedure {

    var input: Pending<String> = .pending
    var output: Pending<ProcedureResult<Profile>> = .pending

    init(session: NetworkSession = URLSession.shared) {

        // Create a URL Request to get the Profile
        let createURLRequest = TransformProcedure<String, URLRequest> { profileId in
            return ProfileAPI(id: profileId).request()
        }

        // Get the Profile
        let download = NetworkProcedure(NetworkDataProcedure(session: session))
            // This line injects the URL Request at the correct time
            .injectResult(from: createURLRequest)

        // Decode JSON response into Profile (assuming Profile conforms to Decodable)
        let decode = DecodeJSONProcedure<Profile>()
            // This line injects the data from the Download
            .injectPayload(fromNetwork: download)

        super.init(operations: [ createURLRequest, download, decode])

        // This line injects the ProfileId value to create the URL Request
        bind(to: createURLRequest) 
        
        // This line sets the output of the whole group to that of the child Decode procedure.
        bind(from: decode) 
    }
}

Once you have this, we effectively have a function (procedure) which does: String -> Profile, so now we just need to make it do [String] -> [Profile]. This is what BatchProcedure does.

Write a procedure to perform the loop

For your example, we can actually just use BatchProcedure directly:

let profileIDs: [String] = ["john", "george", "paul", "ringo"]
let download = BatchProcedure { GetProfileProcedure(session: session) }
    .injectResult(from: ResultProcedure { profileIDs })

However, in many cases we might want to add a "pre-processing" and maybe post-processing stage, its best to wrap using a group.

Conclusion

Basically - you're correct, but taking advantage of result injection can reduce your application's procedures down to a series of one liners which each perform a step in your process.

---

1. DecodeJSONProcedure is on the development branch
2. The NetworkProcedure is a procedure which wraps other networking procedures to make them network resilient, for example, if the network times out, then it will automatically re-try.

[kiwiswift]

Hi @danthorpe,

Thank you so much for the detailed reply and sorry for taking so long to respond, but I had to focus on another part of my project.

I've implemented the approach you suggested and it works perfectly! The only thing I had to tweak was the download procedure definition, I've changed the parenthesis to curly brackets as it was not compiling:

        // Get the Profile
        let download = NetworkProcedure { NetworkDataProcedure(session: session) }
            // This line injects the URL Request at the correct time
            .injectResult(from: createURLRequest)