Firebase Compatibility

[iwanwmys]

Thanks for the chat Michael! This issue relates to using ScramJet on Firebase Cloud Functions, which uses Nodejs 6.

Here's my index.ts file which I'm deploying to Firebase Cloud Functions:

import * as functions from 'firebase-functions';
import * as admin from "firebase-admin";

// Initialize Firebase App
admin.initializeApp()
const gcs = admin.storage();

// This is the function I want to send to Firebase Cloud Functions
export const generateCSV = functions.storage
    .object()
    .onFinalize(async object => { // <--- This is triggered whenever a file is uploaded to the storage bucket
        const bucket = gcs.bucket(object.bucket);
        const filePath = object.name as string;
        
        const { StringStream } = require("scramjet");

        StringStream.pipeline(
            await bucket.file(filePath).createReadStream(),      // fetch csv
            (stream : any) => stream.CSVParse({}) // parse into objects
        )
            .consume(async (row: Object) => await console.log("Row:", row))  // do whatever you like with the objects
            .then(() => console.log("all done"))

            // Note to M: Once I can start logging these rows, I plan on doing a call that will insert this data into a Firestore Database

        return 
    });

When executing the above function, the logs show the following error:

/user_code/node_modules/scramjet/node_modules/scramjet-core/lib/util/promise-transform-stream.js:153
    async whenRead(count) {
          ^^^^^^^^

SyntaxError: Unexpected identifier
    at createScript (vm.js:56:10)
    at Object.runInThisContext (vm.js:97:10)
    at Module._compile (module.js:549:28)
    at Object.Module._extensions..js (module.js:586:10)
    at Module.load (module.js:494:32)
    at tryModuleLoad (module.js:453:12)
    at Function.Module._load (module.js:445:3)
    at Module.require (module.js:504:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (/user_code/node_modules/scramjet/node_modules/scramjet-core/lib/index.js:2:19)

I would LOVE to be able to use ScramJet on Firebase Cloud Functions :)

[MichalCz]

Hi,

Thanks for the issue - as I see it we have two items to address:

• Scramjet is not compatible with node v6.x.
• full TypeScript compatibility is an issue, but only if used as an import, not with require.

The second part does not affect your case, so we'll inspect the first part now and update this issue on that front. As to full TypeScript compatibility, my colleague informed me of how much changes it requires and it's a whole project that I must consider pushing to the next major release.

[iwanwmys]

Of course, thanks so much! Best of luck :)

[MichalCz]

@iwanwmys - I created a separate task for the compatibility issue - you may want to track that one.

There's some progress on running scramjet-core in v6 so it should all be doable. :)

[iwanwmys]

Awesome thanks for the update :)

…

On Sun, Feb 24, 2019 at 8:35 PM Budleigh Salterton ***@***.***> wrote: @iwanwmys <https://github.com/iwanwmys> - I created a separate task for the compatibility issue - you may want to track that one. There's some progress on running scramjet-core in v6 so it should all be doable. :) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#39 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Atq9IY-09VOu27eUQJni8Oj1yW2yERo6ks5vQttlgaJpZM4bJeWm> .

-- *Iwan Pieterse* *Team Leader*Email: iwan@wemanageyoursite.com Twitter: @wemanageyrsite <http://twitter.com/wemanageyrsite>

[MichalCz]

@iwanwmys just to let you know - it's not that easy getting the tests to pass so this will take a while...

[MichalCz]

Hi @iwanwmys - I have tried to finish this, but we're facing a couple issues with this:

1. es6 async/await constructs don't seem to work well - I cannot get some of these to resolve, which means the test processes are stalling or failing.
2. Although AsyncGenerators seem to work, they are not usable - passing an async generator from outside scramjet simply fails.
3. The issue above is fixable, but at an major expense on the performance front which impacts node 8.0 and up.

Additionally Node 6.x series is going out of support phase by end of April... I've spent a couple days trying to sort this out and even dragged in my colleague to do some testing and despite we see a solution - it is time consuming and has major drawbacks. Therefore after assessment I decided we can't re-enable support for node v6.x.

There is one way to still use scramjet and firebase - that's moving to node 8 on firebase which is possible since august last year. If you'd like to try that, please contact me via email - I'd try to help out.

[MichalCz]

Here's a link on how to turn on node v8.x engine: https://howtofirebase.com/cloud-functions-migrating-to-node-8-9640731a8acc

[iwanwmys]

Awesome, thank you so much. Firebase cloud functions use node 6 by default, I wasn't aware you could switch to node 8. Thanks for pointing that out :)

…

On Thu, Mar 21, 2019 at 1:07 PM Budleigh Salterton ***@***.***> wrote: Here's a link on how to turn on node v8.x engine: https://howtofirebase.com/cloud-functions-migrating-to-node-8-9640731a8acc — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#39 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Atq9IR-KRurDCzIN-4Mont6Rf3maxKSWks5vY2ftgaJpZM4bJeWm> .

-- *Iwan Pieterse* *Team Leader*Email: iwan@wemanageyoursite.com Twitter: @wemanageyrsite <http://twitter.com/wemanageyrsite>