最新消息: 电脑我帮您提供丰富的电脑知识,编程学习,软件下载,win7系统下载。

我如何在NodeJS中等待多个异步函数,然后将它们全部退还以更新Firestore文档?

IT培训 admin 15浏览 0评论

我如何在NodeJS中等待多个异步函数,然后将它们全部退还以更新Firestore文档?

我将简化下面的代码,并在整个过程中添加注释。

[通常的想法是,一旦在我的android应用程序中保存了“销售线索”(客户电话)(用于我的pops的建筑演出),我便在一个带有pupeeteer的网站上抓取一个有关该地址的数据。我已经完成所有工作,但是(到目前为止-成功)我一直在从抓取的网站中保存“外部URL”,但是现在我只想获取外部URL,获取图像并将其上传到我的Firestore存储中,获取签名URL,然后使用此“签名URL”(而不是外部URL)更新firestore文档。一切似乎都可以正常工作,只是顺序不正确。我正在获取图片网址,并且已成功将其保存到Firestore存储中,但是我签名的URL回来太晚了。我只是在这里处理回调(或异步函数)而感到困惑。我不一定需要代码(但不会拒绝它!哈哈),也许只是我如何组织代码的一般指南。我已经阅读了许多有关回调,promise等的信息,但是似乎无法绕开我的头和/或将其应用于该特定示例。

谢谢您的帮助!非常感谢!

exports.fetchData = functions.https.onCall(async (data, context) => {
    const urlLib = require("url");
    const pathLib = require("path"); 

    //Check for authentication
    if (!context.auth) { //not authed -> error
        throw new functions.https.HttpsError('failed-precondition', 'The function must be called ' +
            'while authenticated.');
    } else { //authed -> good to go

        var salesLeadId = data.salesLeadId; //grab leadID which is passed from my android app into this function (unique ID to identify the record I need to update)       

        try {    
            //Example URL
            let url = ''; //I left out code here where I build a custom URL etc.

            const browser = await puppeteer.launch(); 
            const [page] = await browser.pages();
            //Emulate Headless Chrome instance        
            await page.emulate({
                name: 'Desktop 1920x1080',
                userAgent: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36',
                viewport: {
                width: 1920,
                height: 1080
                }
            });
            //Opens Page, and waitws until loading is done
            await page.goto(url, { waitUntil: 'networkidle0' });

            //Read Data from web page and return as data object
            var data = await page.evaluate(() => {

                var scrapedPhoto = document.querySelector("#imgExamplePhoto").getAttribute('src'); //This is the external URL to the image
                // [...] grabbing other info here and adding it to "data"
                var accNo = "123456789"; //Example Data but in reality also fetched via document.queryselector
                var pId "12-44-66-88-88"; //Example Data but in reality also fetched via document.queryselector
                // etc...

                return { scrapedPhoto, accNo, pId }; //return all scraped data in var data

            });

            await browser.close();

            //below function is to grab the external URL, save it to Firestore Storage and return a Signed URL which I would then save with my lead document
            var uploadedPhotoUrl = await saveToStorage(data.scrapedPhoto); // THIS IS WHERE MY PROBLEM IS....the code keeps going. The function eventually returns the correct URL but way too late.

            //Firestore Document Reference for record to be updated
            let leadRef = admin.firestore().collection("sales_leads").doc(salesLeadId);

            //I've changed this in every way that I can think of by trial and error (i.e. tried chaining "thens", but unsuccessfully). I assume first I need to resolve the saveStorage function before proceeding to update my firestore document
            return leadRef.update({ //Here is where I'm going back to my Firestore Document and update it with this "new data"
                photo: uploadedPhotoUrl, //Problem: this is undefined because my code does NOT wait on "saveToStorage" above (Previously this was just the external "scrapedPhotoUrl" and thus worked fine. But instead of saving the external URL, I'm trying to save this image to firestore to avoid constant calls to their web page)
                pId: data.pID,
                accNo: data.accNo
            }).then(writeResult => {
                console.log(`Sales Lead updated with Data on: ${writeResult.writeTime.toDate()}`);
                return `Sales Lead updated with Data on: ${writeResult.writeTime.toDate()}`
            });        

        } catch (err) {
            console.error(err);
            return 'Error';
        }


    }
});

async function saveToStorage(fileUrl) {
    var storage = admin.storage();
    var fetch = require("node-fetch");
    var urlLib = require("url");
    var pathLib = require("path");
    //Get File Name from provided URL
    var parsed = urlLib.parse(fileUrl);
    var fileName = pathLib.basename(parsed.pathname);

    //Create Storage Reference with new File Name
    var bucket = storage.bucket('gs://myprojetname.appspot');
    var folderPath = 'sales/photos/';
    var internalFilePath = folderPath + fileName;
    var file = bucket.file(internalFilePath);

    //Fetch file from url
    return await fetch(fileUrl).then(res => {
      const contentType = res.headers.get('content-type');
      const writeStream = file.createWriteStream({
        metadata: {
          contentType
        }
      });
      return res.body.pipe(writeStream)
        .on('finish', () => {
          console.log('Image saved')
          return getDownloadURL(file); //so this eventually returns the corret URL, but by the time this "comes back" my main function has finished execution
        })
        .on('error', err => {
          writeStream.end();
          return console.error(err);
        });

    });


  }

//Get Signed Download URL from Firestore Storage. This works fine for me.
  function getDownloadURL(file) {     
    const config = {
        action: 'read',
        expires: '10-30-2050'
      };

    return file.getSignedUrl(config).then((downloadUrl) => {
      let result = downloadUrl[0];
      return result;
    }).catch(error => {
      console.error(error);
      return 'Error';
    });

  }




回答如下:

所以我认为“我要到达那里”。我意识到这个问题似乎在我的“ saveToStorage”函数中,如我的日志记录输出所示。 (console.log在结果返回到那里之前发布)。

现在,我从根本上(我认为)以正确的顺序进行操作。

saveToStorage函数=>

[...]
  return await fetch(fileUrl).then(res => {
    const contentType = res.headers.get('content-type');
    const writeStream = file.createWriteStream({
      metadata: {
        contentType
      }
    });
    let p = new Promise((resolve, reject) => {
      res.body.pipe(writeStream).on('finish', function() {
        resolve(file);
      });
      res.body.pipe(writeStream).on('error', function () {
        reject();
      });
    });

    return p.then(function(file) {
      console.log('Image saved')
      return getDownloadURL(file);
    });

  });
[...]

我在此函数中添加了一个Promise,在返回数据之前要对其进行解析。我得到正确的URL,并且正确的存储(签名)URL已写入/更新到我的Firestore文档中。...但是...

我的图像只写了一半。下半部分是灰色像素。 :head scratching:(以前,图像很好,但是返回的URL太晚了。现在,URL被“及时”返回,但是图像似乎不完整。)

编辑:我将其标记为答案,并用我的新问题打开一个新问题。我相信我原来的问题已经解决了。 (编辑:明天)

我如何在NodeJS中等待多个异步函数,然后将它们全部退还以更新Firestore文档?

我将简化下面的代码,并在整个过程中添加注释。

[通常的想法是,一旦在我的android应用程序中保存了“销售线索”(客户电话)(用于我的pops的建筑演出),我便在一个带有pupeeteer的网站上抓取一个有关该地址的数据。我已经完成所有工作,但是(到目前为止-成功)我一直在从抓取的网站中保存“外部URL”,但是现在我只想获取外部URL,获取图像并将其上传到我的Firestore存储中,获取签名URL,然后使用此“签名URL”(而不是外部URL)更新firestore文档。一切似乎都可以正常工作,只是顺序不正确。我正在获取图片网址,并且已成功将其保存到Firestore存储中,但是我签名的URL回来太晚了。我只是在这里处理回调(或异步函数)而感到困惑。我不一定需要代码(但不会拒绝它!哈哈),也许只是我如何组织代码的一般指南。我已经阅读了许多有关回调,promise等的信息,但是似乎无法绕开我的头和/或将其应用于该特定示例。

谢谢您的帮助!非常感谢!

exports.fetchData = functions.https.onCall(async (data, context) => {
    const urlLib = require("url");
    const pathLib = require("path"); 

    //Check for authentication
    if (!context.auth) { //not authed -> error
        throw new functions.https.HttpsError('failed-precondition', 'The function must be called ' +
            'while authenticated.');
    } else { //authed -> good to go

        var salesLeadId = data.salesLeadId; //grab leadID which is passed from my android app into this function (unique ID to identify the record I need to update)       

        try {    
            //Example URL
            let url = ''; //I left out code here where I build a custom URL etc.

            const browser = await puppeteer.launch(); 
            const [page] = await browser.pages();
            //Emulate Headless Chrome instance        
            await page.emulate({
                name: 'Desktop 1920x1080',
                userAgent: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36',
                viewport: {
                width: 1920,
                height: 1080
                }
            });
            //Opens Page, and waitws until loading is done
            await page.goto(url, { waitUntil: 'networkidle0' });

            //Read Data from web page and return as data object
            var data = await page.evaluate(() => {

                var scrapedPhoto = document.querySelector("#imgExamplePhoto").getAttribute('src'); //This is the external URL to the image
                // [...] grabbing other info here and adding it to "data"
                var accNo = "123456789"; //Example Data but in reality also fetched via document.queryselector
                var pId "12-44-66-88-88"; //Example Data but in reality also fetched via document.queryselector
                // etc...

                return { scrapedPhoto, accNo, pId }; //return all scraped data in var data

            });

            await browser.close();

            //below function is to grab the external URL, save it to Firestore Storage and return a Signed URL which I would then save with my lead document
            var uploadedPhotoUrl = await saveToStorage(data.scrapedPhoto); // THIS IS WHERE MY PROBLEM IS....the code keeps going. The function eventually returns the correct URL but way too late.

            //Firestore Document Reference for record to be updated
            let leadRef = admin.firestore().collection("sales_leads").doc(salesLeadId);

            //I've changed this in every way that I can think of by trial and error (i.e. tried chaining "thens", but unsuccessfully). I assume first I need to resolve the saveStorage function before proceeding to update my firestore document
            return leadRef.update({ //Here is where I'm going back to my Firestore Document and update it with this "new data"
                photo: uploadedPhotoUrl, //Problem: this is undefined because my code does NOT wait on "saveToStorage" above (Previously this was just the external "scrapedPhotoUrl" and thus worked fine. But instead of saving the external URL, I'm trying to save this image to firestore to avoid constant calls to their web page)
                pId: data.pID,
                accNo: data.accNo
            }).then(writeResult => {
                console.log(`Sales Lead updated with Data on: ${writeResult.writeTime.toDate()}`);
                return `Sales Lead updated with Data on: ${writeResult.writeTime.toDate()}`
            });        

        } catch (err) {
            console.error(err);
            return 'Error';
        }


    }
});

async function saveToStorage(fileUrl) {
    var storage = admin.storage();
    var fetch = require("node-fetch");
    var urlLib = require("url");
    var pathLib = require("path");
    //Get File Name from provided URL
    var parsed = urlLib.parse(fileUrl);
    var fileName = pathLib.basename(parsed.pathname);

    //Create Storage Reference with new File Name
    var bucket = storage.bucket('gs://myprojetname.appspot');
    var folderPath = 'sales/photos/';
    var internalFilePath = folderPath + fileName;
    var file = bucket.file(internalFilePath);

    //Fetch file from url
    return await fetch(fileUrl).then(res => {
      const contentType = res.headers.get('content-type');
      const writeStream = file.createWriteStream({
        metadata: {
          contentType
        }
      });
      return res.body.pipe(writeStream)
        .on('finish', () => {
          console.log('Image saved')
          return getDownloadURL(file); //so this eventually returns the corret URL, but by the time this "comes back" my main function has finished execution
        })
        .on('error', err => {
          writeStream.end();
          return console.error(err);
        });

    });


  }

//Get Signed Download URL from Firestore Storage. This works fine for me.
  function getDownloadURL(file) {     
    const config = {
        action: 'read',
        expires: '10-30-2050'
      };

    return file.getSignedUrl(config).then((downloadUrl) => {
      let result = downloadUrl[0];
      return result;
    }).catch(error => {
      console.error(error);
      return 'Error';
    });

  }




回答如下:

所以我认为“我要到达那里”。我意识到这个问题似乎在我的“ saveToStorage”函数中,如我的日志记录输出所示。 (console.log在结果返回到那里之前发布)。

现在,我从根本上(我认为)以正确的顺序进行操作。

saveToStorage函数=>

[...]
  return await fetch(fileUrl).then(res => {
    const contentType = res.headers.get('content-type');
    const writeStream = file.createWriteStream({
      metadata: {
        contentType
      }
    });
    let p = new Promise((resolve, reject) => {
      res.body.pipe(writeStream).on('finish', function() {
        resolve(file);
      });
      res.body.pipe(writeStream).on('error', function () {
        reject();
      });
    });

    return p.then(function(file) {
      console.log('Image saved')
      return getDownloadURL(file);
    });

  });
[...]

我在此函数中添加了一个Promise,在返回数据之前要对其进行解析。我得到正确的URL,并且正确的存储(签名)URL已写入/更新到我的Firestore文档中。...但是...

我的图像只写了一半。下半部分是灰色像素。 :head scratching:(以前,图像很好,但是返回的URL太晚了。现在,URL被“及时”返回,但是图像似乎不完整。)

编辑:我将其标记为答案,并用我的新问题打开一个新问题。我相信我原来的问题已经解决了。 (编辑:明天)

发布评论

评论列表 (0)

  1. 暂无评论