Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Markdown files cannot be found in netlify function environment #153

Closed
MejanH opened this issue Dec 13, 2020 · 19 comments
Closed

Markdown files cannot be found in netlify function environment #153

MejanH opened this issue Dec 13, 2020 · 19 comments
Labels
priority: high type: bug code to address defects in shipped code

Comments

@MejanH
Copy link

MejanH commented Dec 13, 2020

Hi. First of all, Thanks for the plugin.

I am getting errors with my markdown blog. Pages and components seem to be working perfectly (tested with sample). but not working with the markdown directory. It's giving me an error related to my file location.

{
"errorType": "Runtime.UnhandledPromiseRejection",
"errorMessage": "Error: ENOENT: no such file or directory, scandir '/var/task/contents/blog'",
"trace": [
"Runtime.UnhandledPromiseRejection: Error: ENOENT: no such file or directory, scandir '/var/task/contents/blog'",
"    at process.<anonymous> (/var/runtime/index.js:35:15)",
"    at process.emit (events.js:314:20)",
"    at processPromiseRejections (internal/process/promises.js:209:33)",
"    at processTicksAndRejections (internal/process/task_queues.js:98:32)"
]
}

Deploy Log

5:36:18 PM: Build ready to start
5:36:20 PM: build-image version: 53b83b6bede2920f236b25b6f5a95334320dc849
5:36:20 PM: build-image tag: v3.6.0
5:36:20 PM: buildbot version: 94ed42511c4e70547a960237db686b74875fef2d
5:36:20 PM: Fetching cached dependencies
5:36:20 PM: Starting to download cache of 275.3KB
5:36:20 PM: Finished downloading cache in 77.823649ms
5:36:20 PM: Starting to extract cache
5:36:20 PM: Finished extracting cache in 14.676215ms
5:36:20 PM: Finished fetching cache in 92.929933ms
5:36:20 PM: Starting to prepare the repo for build
5:36:21 PM: Preparing Git Reference refs/heads/main
5:36:23 PM: Starting build script
5:36:23 PM: Installing dependencies
5:36:23 PM: Python version set to 2.7
5:36:24 PM: v12.18.0 is already installed.
5:36:25 PM: Now using node v12.18.0 (npm v6.14.4)
5:36:25 PM: Started restoring cached build plugins
5:36:25 PM: Finished restoring cached build plugins
5:36:25 PM: Attempting ruby version 2.7.1, read from environment
5:36:27 PM: Using ruby version 2.7.1
5:36:27 PM: Using PHP version 5.6
5:36:27 PM: Started restoring cached node modules
5:36:27 PM: Finished restoring cached node modules
5:36:27 PM: Started restoring cached yarn cache
5:36:27 PM: Finished restoring cached yarn cache
5:36:27 PM: Installing yarn at version 1.22.4
5:36:27 PM: Installing Yarn!
5:36:27 PM: > Downloading tarball...
5:36:27 PM: [1/2]: https://yarnpkg.com/downloads/1.22.4/yarn-v1.22.4.tar.gz --> /tmp/yarn.tar.gz.OaqmPaAqag
5:36:27 PM:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
5:36:27 PM:                                  Dload  Upload   Total   Spent    Left  Speed
5:36:28 PM:   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
5:36:28 PM: 100    80  100    80    0     0    241      0 --:--:-- --:--:-- --:--:--   241
5:36:28 PM:   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
5:36:28 PM: 100    93  100    93    0     0    138      0 --:--:-- --:--:-- --:--:--  3444
5:36:28 PM: 100   643  100   643    0     0    620      0  0:00:01  0:00:01 --:--:--   620
5:36:29 PM: 100 1215k  100 1215k    0     0   802k      0  0:00:01  0:00:01 --:--:--  802k
5:36:29 PM: [2/2]: https://yarnpkg.com/downloads/1.22.4/yarn-v1.22.4.tar.gz.asc --> /tmp/yarn.tar.gz.OaqmPaAqag.asc
5:36:29 PM: 100    84  100    84    0     0   3073      0 --:--:-- --:--:-- --:--:--  3073
5:36:29 PM: 100    97  100    97    0     0   1764      0 --:--:-- --:--:-- --:--:--  1764
5:36:29 PM:   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
5:36:29 PM: 100   647  100   647    0     0   3829      0 --:--:-- --:--:-- --:--:--  631k
5:36:29 PM: 100  1028  100  1028    0     0   4923      0 --:--:-- --:--:-- --:--:--  4923
5:36:29 PM: > Verifying integrity...
5:36:29 PM: gpg: Signature made Mon 09 Mar 2020 03:52:13 PM UTC using RSA key ID 69475BAA
5:36:29 PM: gpg: Good signature from "Yarn Packaging <yarn@dan.cx>"
5:36:29 PM: gpg: WARNING: This key is not certified with a trusted signature!
5:36:29 PM: gpg:          There is no indication that the signature belongs to the owner.
5:36:29 PM: Primary key fingerprint: 72EC F46A 56B4 AD39 C907  BBB7 1646 B01B 86E5 0310
5:36:29 PM:      Subkey fingerprint: 6D98 490C 6F1A CDDD 448E  4595 4F77 6793 6947 5BAA
5:36:29 PM: > GPG signature looks good
5:36:29 PM: > Extracting to ~/.yarn...
5:36:29 PM: > Adding to $PATH...
5:36:30 PM: > Successfully installed Yarn 1.22.4! Please open another terminal where the `yarn` command will now be available.
5:36:30 PM: Installing NPM modules using Yarn version 1.22.4
5:36:30 PM: yarn install v1.22.4
5:36:30 PM: [1/4] Resolving packages...
5:36:31 PM: [2/4] Fetching packages...
5:36:42 PM: info fsevents@2.1.3: The platform "linux" is incompatible with this module.
5:36:42 PM: info "fsevents@2.1.3" is an optional dependency and failed compatibility check. Excluding it from installation.
5:36:42 PM: info fsevents@1.2.13: The platform "linux" is incompatible with this module.
5:36:42 PM: info "fsevents@1.2.13" is an optional dependency and failed compatibility check. Excluding it from installation.
5:36:42 PM: [3/4] Linking dependencies...
5:36:46 PM: [4/4] Building fresh packages...
5:36:50 PM: Done in 19.73s.
5:36:50 PM: NPM modules installed using Yarn
5:36:50 PM: Started restoring cached go cache
5:36:50 PM: Finished restoring cached go cache
5:36:50 PM: go version go1.14.4 linux/amd64
5:36:50 PM: go version go1.14.4 linux/amd64
5:36:50 PM: Installing missing commands
5:36:50 PM: Verify run directory
5:36:52 PM: ​
5:36:52 PM: ────────────────────────────────────────────────────────────────
5:36:52 PM:   Netlify Build                                                 
5:36:52 PM: ────────────────────────────────────────────────────────────────
5:36:52 PM: ​
5:36:52 PM: ❯ Version
5:36:52 PM:   @netlify/build 7.0.0
5:36:52 PM: ​
5:36:52 PM: ❯ Flags
5:36:52 PM:   deployId: 5fd5fcb20cd506b6ee26c0bc
5:36:52 PM:   mode: buildbot
5:36:52 PM: ​
5:36:52 PM: ❯ Current directory
5:36:52 PM:   /opt/build/repo
5:36:52 PM: ​
5:36:52 PM: ❯ Config file
5:36:52 PM:   No config file was defined: using default values.
5:36:52 PM: ​
5:36:52 PM: ❯ Context
5:36:52 PM:   production
5:36:52 PM: ​
5:36:52 PM: ❯ Installing plugins
5:36:52 PM:    - @netlify/plugin-nextjs@1.0.2
5:37:25 PM: ​
5:37:25 PM: ❯ Loading plugins
5:37:25 PM:    - @netlify/plugin-nextjs@1.0.2 from Netlify app
5:37:26 PM: ​
5:37:26 PM: ────────────────────────────────────────────────────────────────
5:37:26 PM:   1. onPreBuild command from @netlify/plugin-nextjs             
5:37:26 PM: ────────────────────────────────────────────────────────────────
5:37:26 PM: ​
5:37:26 PM: ** Adding next.config.js with target set to 'serverless' **
5:37:26 PM: ​
5:37:26 PM: (@netlify/plugin-nextjs onPreBuild completed in 10ms)
5:37:26 PM: ​
5:37:26 PM: ────────────────────────────────────────────────────────────────
5:37:26 PM:   2. Build command from Netlify app                             
5:37:26 PM: ────────────────────────────────────────────────────────────────
5:37:26 PM: ​
5:37:26 PM: $ yarn build
5:37:26 PM: yarn run v1.22.4
5:37:26 PM: $ next build
5:37:27 PM: warn  - No build cache found. Please configure build caching for faster rebuilds. Read more: https://err.sh/next.js/no-cache
5:37:27 PM: info  - Creating an optimized production build...
5:37:51 PM: warn  - Compiled with warnings
5:37:51 PM: ./node_modules/next/dist/next-server/server/load-components.js
5:37:51 PM: Critical dependency: the request of a dependency is an expression
5:37:51 PM: ./node_modules/next/dist/next-server/server/load-components.js
5:37:51 PM: Critical dependency: the request of a dependency is an expression
5:37:51 PM: ./node_modules/next/dist/next-server/server/require.js
5:37:51 PM: Critical dependency: the request of a dependency is an expression
5:37:51 PM: ./node_modules/next/dist/next-server/server/require.js
5:37:51 PM: Critical dependency: the request of a dependency is an expression
5:37:51 PM: ./node_modules/next/dist/next-server/server/require.js
5:37:51 PM: Critical dependency: the request of a dependency is an expression
5:37:51 PM: ./node_modules/next/node_modules/node-fetch/lib/index.js
5:37:51 PM: Module not found: Can't resolve 'encoding' in '/opt/build/repo/node_modules/next/node_modules/node-fetch/lib'
5:37:51 PM: info  - Collecting page data...
5:37:52 PM: info  - Generating static pages (0/4)
5:37:53 PM: info  - Generating static pages (1/4)
5:37:53 PM: info  - Generating static pages (2/4)
5:37:53 PM: info  - Generating static pages (3/4)
5:37:53 PM: info  - Generating static pages (4/4)
5:37:53 PM: info  - Finalizing page optimization...
5:37:53 PM: Page                                Size     First Load JS
5:37:53 PM: ┌ ● / (ISR: 1 Seconds)              1.97 kB        64.3 kB
5:37:53 PM: ├   /_app                           0 B            62.3 kB
5:37:53 PM: ├ ○ /404                            3.01 kB        65.4 kB
5:37:53 PM: ├ λ /api/hello                      0 B            62.3 kB
5:37:53 PM: └ ● /blog/[id]                      1.94 kB        64.3 kB
5:37:53 PM:     ├ /blog/static-site-generation
5:37:53 PM:     └ /blog/test-post
5:37:53 PM: + First Load JS shared by all       62.3 kB
5:37:53 PM:   ├ chunks/commons.40559a.js        12.8 kB
5:37:53 PM:   ├ chunks/framework.ef157c.js      41.8 kB
5:37:53 PM:   ├ chunks/main.c60f4c.js           6.54 kB
5:37:53 PM:   ├ chunks/pages/_app.e774f4.js     526 B
5:37:53 PM:   ├ chunks/webpack.e06743.js        751 B
5:37:53 PM:   └ css/6fcf17ab8077bd559672.css    2.31 kB
5:37:53 PM: λ  (Lambda)  server-side renders at runtime (uses getInitialProps or getServerSideProps)
5:37:53 PM: ○  (Static)  automatically rendered as static HTML (uses no initial props)
5:37:53 PM: ●  (SSG)     automatically generated as static HTML + JSON (uses getStaticProps)
5:37:53 PM:    (ISR)     incremental static regeneration (uses revalidate in getStaticProps)
5:37:53 PM: Done in 26.75s.
5:37:53 PM: ​
5:37:53 PM: (build.command completed in 27s)
5:37:53 PM: ​
5:37:53 PM: ────────────────────────────────────────────────────────────────
5:37:53 PM:   3. onBuild command from @netlify/plugin-nextjs                
5:37:53 PM: ────────────────────────────────────────────────────────────────
5:37:53 PM: ​
5:37:53 PM: ** Running Next on Netlify package **
5:37:53 PM: 🚀 Next on Netlify 🚀
5:37:53 PM:    Functions directory:  netlify-automatic-functions
5:37:53 PM:    Publish directory:  /opt/build/repo
5:37:53 PM:    Make sure these are set in your netlify.toml file.
5:37:53 PM: 🌍️ Copying public/ folder to /opt/build/repo
5:37:53 PM: 💼 Copying static NextJS assets to /opt/build/repo
5:37:53 PM: 💫 Setting up API endpoints as Netlify Functions in netlify-automatic-functions
5:37:53 PM: 💫 Setting up pages with getInitialProps as Netlify Functions in netlify-automatic-functions
5:37:53 PM: 💫 Setting up pages with getServerSideProps as Netlify Functions in netlify-automatic-functions
5:37:53 PM: 🔥 Copying pre-rendered pages with getStaticProps and JSON data to /opt/build/repo
5:37:53 PM: 💫 Setting up pages with getStaticProps and fallback: true as Netlify Functions in netlify-automatic-functions
5:37:53 PM: 💫 Setting up pages with getStaticProps and revalidation interval as Netlify Functions in netlify-automatic-functions
5:37:53 PM: 🔥 Copying pre-rendered pages without props to /opt/build/repo
5:37:53 PM: 🔀 Setting up redirects
5:37:53 PM: ​
5:37:53 PM: (@netlify/plugin-nextjs onBuild completed in 167ms)
5:37:53 PM: ​
5:37:53 PM: ────────────────────────────────────────────────────────────────
5:37:53 PM:   4. Functions bundling                                         
5:37:53 PM: ────────────────────────────────────────────────────────────────
5:37:53 PM: ​
5:37:53 PM: Packaging Functions from netlify-automatic-functions directory:
5:37:53 PM:  - next_api_hello/next_api_hello.js
5:37:53 PM:  - next_blog_id/next_blog_id.js
5:37:53 PM:  - next_index/next_index.js
5:38:05 PM: ​
5:38:05 PM: (Functions bundling completed in 11.9s)
5:38:05 PM: ​
5:38:05 PM: ────────────────────────────────────────────────────────────────
5:38:05 PM:   Netlify Build Complete                                        
5:38:05 PM: ────────────────────────────────────────────────────────────────
5:38:05 PM: ​
5:38:05 PM: (Netlify Build completed in 1m 12.9s)
5:38:05 PM: Caching artifacts
5:38:05 PM: Started saving node modules
5:38:05 PM: Finished saving node modules
5:38:05 PM: Started saving build plugins
5:38:05 PM: Finished saving build plugins
5:38:05 PM: Started saving yarn cache
5:38:05 PM: Finished saving yarn cache
5:38:05 PM: Started saving pip cache
5:38:06 PM: Finished saving pip cache
5:38:06 PM: Started saving emacs cask dependencies
5:38:06 PM: Finished saving emacs cask dependencies
5:38:06 PM: Started saving maven dependencies
5:38:06 PM: Finished saving maven dependencies
5:38:06 PM: Started saving boot dependencies
5:38:06 PM: Finished saving boot dependencies
5:38:06 PM: Started saving rust rustup cache
5:38:06 PM: Finished saving rust rustup cache
5:38:06 PM: Started saving rust cargo bin cache
5:38:06 PM: Finished saving rust cargo bin cache
5:38:06 PM: Started saving go dependencies
5:38:06 PM: Finished saving go dependencies
5:38:08 PM: Build script success
5:38:08 PM: Starting to deploy site from ''
5:38:08 PM: Creating deploy tree 
5:38:09 PM: Creating deploy upload records
5:38:09 PM: 5 new files to upload
5:38:09 PM: 3 new functions to upload
5:38:10 PM: Starting post processing
5:38:12 PM: Post processing - HTML
5:38:12 PM: Post processing - header rules
5:38:12 PM: Post processing - redirect rules
5:38:12 PM: Post processing done
5:38:13 PM: Site is live ✨
5:38:34 PM: Finished processing build request in 2m14.274321422s
@lindsaylevine
Copy link

hey @MejanH thanks for opening this! happy to help. however, need a little bit more info from you. the fastest/easiest way for me to help you is for you to open a repo that reproduces this issue and that i can deploy myself. from the deploy logs you shared, i don't see a markdown dir under pages which means, wherever you have it, it's excluded from what next-on-netlify or next as a framework would handle. let me know!

@MejanH
Copy link
Author

MejanH commented Dec 14, 2020

Thanks for the reply. For the demo, I just followed the next.js learn pages. At build time there is no problem with 'fs". But I don't why it's not working on view.

The completed lesson repo

Next.js Learn Page

@lindsaylevine
Copy link

hey, i spent a bit of time trying to get typescript-final working on my machine. that repo seems somewhat broken. when you initially run next dev, you get an invalid date error. then next build fails trying to look for files ending in .md.html. because this doesn't even work locally for me, there's not much i can do.

also, i can tell from your logs in your original post that what you have deviates from typescript-final in at least some way. can you please open a repo with exactly what you tried to deploy to netlify?

@MejanH
Copy link
Author

MejanH commented Dec 15, 2020

Hi, I have created a repo as you said. tried with npm run build and next build as this official post says. Still the same error. It gets published successfully but not working.

My repo

7:50:06 PM: Build ready to start
7:50:08 PM: build-image version: 53b83b6bede2920f236b25b6f5a95334320dc849
7:50:08 PM: build-image tag: v3.6.0
7:50:08 PM: buildbot version: cdcf5e6d038c4a5c0bbc618d40b7eb21a2893d7d
7:50:08 PM: Building without cache
7:50:08 PM: Starting to prepare the repo for build
7:50:08 PM: No cached dependencies found. Cloning fresh repo
7:50:08 PM: git clone https://github.com/MejanH/nextjs-blog-incremental-static
7:50:09 PM: Preparing Git Reference refs/heads/main
7:50:11 PM: Starting build script
7:50:11 PM: Installing dependencies
7:50:11 PM: Python version set to 2.7
7:50:13 PM: v12.18.0 is already installed.
7:50:13 PM: Now using node v12.18.0 (npm v6.14.4)
7:50:13 PM: Started restoring cached build plugins
7:50:13 PM: Finished restoring cached build plugins
7:50:14 PM: Attempting ruby version 2.7.1, read from environment
7:50:16 PM: Using ruby version 2.7.1
7:50:16 PM: Using PHP version 5.6
7:50:16 PM: Started restoring cached node modules
7:50:16 PM: Finished restoring cached node modules
7:50:16 PM: Installing NPM modules using NPM version 6.14.4
7:50:32 PM: > sharp@0.26.2 install /opt/build/repo/node_modules/sharp
7:50:32 PM: > (node install/libvips && node install/dll-copy && prebuild-install) || (node-gyp rebuild && node install/dll-copy)
7:50:34 PM: info sharp Downloading https://github.com/lovell/sharp-libvips/releases/download/v8.10.0/libvips-8.10.0-linux-x64.tar.br
7:50:37 PM: > @ampproject/toolbox-optimizer@2.7.0-alpha.1 postinstall /opt/build/repo/node_modules/@ampproject/toolbox-optimizer
7:50:37 PM: > node lib/warmup.js
7:50:38 PM: AMP OPTIMIZER Downloaded latest AMP runtime data.
7:50:38 PM: npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.13 (node_modules/watchpack-chokidar2/node_modules/fsevents):
7:50:38 PM: npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.13: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
7:50:38 PM: npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@2.1.3 (node_modules/fsevents):
7:50:38 PM: npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.1.3: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
7:50:38 PM: added 714 packages from 467 contributors and audited 716 packages in 20.898s
7:50:39 PM: 81 packages are looking for funding
7:50:39 PM:   run `npm fund` for details
7:50:39 PM: found 2 low severity vulnerabilities
7:50:39 PM:   run `npm audit fix` to fix them, or `npm audit` for details
7:50:39 PM: NPM modules installed
7:50:39 PM: Started restoring cached go cache
7:50:39 PM: Finished restoring cached go cache
7:50:39 PM: go version go1.14.4 linux/amd64
7:50:39 PM: go version go1.14.4 linux/amd64
7:50:39 PM: Installing missing commands
7:50:39 PM: Verify run directory
7:50:42 PM: ​
7:50:42 PM: ────────────────────────────────────────────────────────────────
7:50:42 PM:   Netlify Build                                                 
7:50:42 PM: ────────────────────────────────────────────────────────────────
7:50:42 PM: ​
7:50:42 PM: ❯ Version
7:50:42 PM:   @netlify/build 8.0.0
7:50:42 PM: ​
7:50:42 PM: ❯ Flags
7:50:42 PM:   deployId: 5fd8bf0e213e5d0054b2fb66
7:50:42 PM:   mode: buildbot
7:50:42 PM: ​
7:50:42 PM: ❯ Current directory
7:50:42 PM:   /opt/build/repo
7:50:42 PM: ​
7:50:42 PM: ❯ Config file
7:50:42 PM:   No config file was defined: using default values.
7:50:42 PM: ​
7:50:42 PM: ❯ Context
7:50:42 PM:   production
7:50:42 PM: ​
7:50:42 PM: ❯ Installing plugins
7:50:42 PM:    - @netlify/plugin-nextjs@1.0.2
7:51:21 PM: ​
7:51:21 PM: ❯ Loading plugins
7:51:21 PM:    - @netlify/plugin-nextjs@1.0.2 from Netlify app
7:51:22 PM: ​
7:51:22 PM: ────────────────────────────────────────────────────────────────
7:51:22 PM:   1. onPreBuild command from @netlify/plugin-nextjs             
7:51:22 PM: ────────────────────────────────────────────────────────────────
7:51:22 PM: ​
7:51:23 PM: ** Adding next.config.js with target set to 'serverless' **
7:51:23 PM: ​
7:51:23 PM: (@netlify/plugin-nextjs onPreBuild completed in 14ms)
7:51:23 PM: ​
7:51:23 PM: ────────────────────────────────────────────────────────────────
7:51:23 PM:   2. Build command from Netlify app                             
7:51:23 PM: ────────────────────────────────────────────────────────────────
7:51:23 PM: ​
7:51:23 PM: $ next build
7:51:24 PM: warn  - No build cache found. Please configure build caching for faster rebuilds. Read more: https://err.sh/next.js/no-cache
7:51:24 PM: info  - Creating an optimized production build...
7:51:43 PM: warn  - Compiled with warnings
7:51:43 PM: ./node_modules/next/dist/next-server/server/load-components.js
7:51:43 PM: Critical dependency: the request of a dependency is an expression
7:51:43 PM: ./node_modules/next/dist/next-server/server/load-components.js
7:51:43 PM: Critical dependency: the request of a dependency is an expression
7:51:43 PM: ./node_modules/next/dist/next-server/server/require.js
7:51:43 PM: Critical dependency: the request of a dependency is an expression
7:51:43 PM: ./node_modules/next/dist/next-server/server/require.js
7:51:43 PM: Critical dependency: the request of a dependency is an expression
7:51:43 PM: ./node_modules/next/dist/next-server/server/require.js
7:51:43 PM: Critical dependency: the request of a dependency is an expression
7:51:43 PM: ./node_modules/node-fetch/lib/index.js
7:51:43 PM: Module not found: Can't resolve 'encoding' in '/opt/build/repo/node_modules/node-fetch/lib'
7:51:43 PM: info  - Collecting page data...
7:51:44 PM: info  - Generating static pages (0/5)
7:51:45 PM: info  - Generating static pages (1/5)
7:51:45 PM: info  - Generating static pages (2/5)
7:51:45 PM: info  - Generating static pages (3/5)
7:51:45 PM: info  - Generating static pages (5/5)
7:51:45 PM: info  - Finalizing page optimization...
7:51:46 PM: Page                              Size     First Load JS
7:51:46 PM: ┌ ● / (ISR: 1 Seconds)            631 B          71.3 kB
7:51:46 PM: ├   /_app                         0 B            60.2 kB
7:51:46 PM: ├ ○ /404                          2.75 kB        62.9 kB
7:51:46 PM: ├ λ /api/hello                    0 B            60.2 kB
7:51:46 PM: └ ● /posts/[id]                   433 B          71.1 kB
7:51:46 PM:     ├ /posts/pre-rendering
7:51:46 PM:     ├ /posts/ssg-ssr
7:51:46 PM:     └ /posts/testing-ssg
7:51:46 PM: + First Load JS shared by all     60.2 kB
7:51:46 PM:   ├ chunks/commons.87066d.js      13 kB
7:51:46 PM:   ├ chunks/framework.9ec1f7.js    39.9 kB
7:51:46 PM:   ├ chunks/main.bb7417.js         6.22 kB
7:51:46 PM:   ├ chunks/pages/_app.ec7c07.js   296 B
7:51:46 PM:   ├ chunks/webpack.e06743.js      751 B
7:51:46 PM:   └ css/2fd5d76e8788e1d11cbb.css  264 B
7:51:46 PM: λ  (Lambda)  server-side renders at runtime (uses getInitialProps or getServerSideProps)
7:51:46 PM: ○  (Static)  automatically rendered as static HTML (uses no initial props)
7:51:46 PM: ●  (SSG)     automatically generated as static HTML + JSON (uses getStaticProps)
7:51:46 PM:    (ISR)     incremental static regeneration (uses revalidate in getStaticProps)
7:51:46 PM: ​
7:51:46 PM: (build.command completed in 23.1s)
7:51:46 PM: ​
7:51:46 PM: ────────────────────────────────────────────────────────────────
7:51:46 PM:   3. onBuild command from @netlify/plugin-nextjs                
7:51:46 PM: ────────────────────────────────────────────────────────────────
7:51:46 PM: ​
7:51:46 PM: ** Running Next on Netlify package **
7:51:46 PM: 🚀 Next on Netlify 🚀
7:51:46 PM:    Functions directory:  netlify-automatic-functions
7:51:46 PM:    Publish directory:  /opt/build/repo
7:51:46 PM:    Make sure these are set in your netlify.toml file.
7:51:46 PM: 🌍️ Copying public/ folder to /opt/build/repo
7:51:46 PM: 💼 Copying static NextJS assets to /opt/build/repo
7:51:46 PM: 💫 Setting up API endpoints as Netlify Functions in netlify-automatic-functions
7:51:46 PM: 💫 Setting up pages with getInitialProps as Netlify Functions in netlify-automatic-functions
7:51:46 PM: 💫 Setting up pages with getServerSideProps as Netlify Functions in netlify-automatic-functions
7:51:46 PM: 🔥 Copying pre-rendered pages with getStaticProps and JSON data to /opt/build/repo
7:51:46 PM: 💫 Setting up pages with getStaticProps and fallback: true as Netlify Functions in netlify-automatic-functions
7:51:46 PM: 💫 Setting up pages with getStaticProps and revalidation interval as Netlify Functions in netlify-automatic-functions
7:51:46 PM: 🔥 Copying pre-rendered pages without props to /opt/build/repo
7:51:46 PM: 🔀 Setting up redirects
7:51:46 PM: ​
7:51:46 PM: (@netlify/plugin-nextjs onBuild completed in 127ms)
7:51:46 PM: ​
7:51:46 PM: ────────────────────────────────────────────────────────────────
7:51:46 PM:   4. Functions bundling                                         
7:51:46 PM: ────────────────────────────────────────────────────────────────
7:51:46 PM: ​
7:51:46 PM: Packaging Functions from netlify-automatic-functions directory:
7:51:46 PM:  - next_api_hello/next_api_hello.js
7:51:46 PM:  - next_index/next_index.js
7:51:46 PM:  - next_posts_id/next_posts_id.js
7:52:00 PM: ​
7:52:00 PM: (Functions bundling completed in 14.3s)
7:52:00 PM: ​
7:52:00 PM: ────────────────────────────────────────────────────────────────
7:52:00 PM:   Netlify Build Complete                                        
7:52:00 PM: ────────────────────────────────────────────────────────────────
7:52:00 PM: ​
7:52:00 PM: (Netlify Build completed in 1m 18.6s)
7:52:01 PM: Caching artifacts
7:52:01 PM: Started saving node modules
7:52:01 PM: Finished saving node modules
7:52:01 PM: Started saving build plugins
7:52:01 PM: Finished saving build plugins
7:52:01 PM: Started saving pip cache
7:52:01 PM: Finished saving pip cache
7:52:01 PM: Started saving emacs cask dependencies
7:52:01 PM: Finished saving emacs cask dependencies
7:52:01 PM: Started saving maven dependencies
7:52:01 PM: Finished saving maven dependencies
7:52:01 PM: Started saving boot dependencies
7:52:01 PM: Finished saving boot dependencies
7:52:01 PM: Started saving rust rustup cache
7:52:01 PM: Finished saving rust rustup cache
7:52:01 PM: Started saving rust cargo bin cache
7:52:01 PM: Finished saving rust cargo bin cache
7:52:01 PM: Started saving go dependencies
7:52:01 PM: Finished saving go dependencies
7:52:04 PM: Build script success
7:52:04 PM: Starting to deploy site from ''
7:52:04 PM: Creating deploy tree 
7:52:04 PM: Creating deploy upload records
7:52:05 PM: 5 new files to upload
7:52:05 PM: 3 new functions to upload
7:52:06 PM: Starting post processing
7:52:06 PM: Post processing - HTML
7:52:07 PM: Post processing - header rules
7:52:07 PM: Post processing - redirect rules
7:52:07 PM: Post processing done
7:52:07 PM: Site is live ✨
7:52:28 PM: Finished processing build request in 2m19.653681844s

JSON error after publishing

{
"errorType": "Runtime.UnhandledPromiseRejection",
"errorMessage": "Error: ENOENT: no such file or directory, scandir '/var/task/posts'",
"trace": [
"Runtime.UnhandledPromiseRejection: Error: ENOENT: no such file or directory, scandir '/var/task/posts'",
"    at process.<anonymous> (/var/runtime/index.js:35:15)",
"    at process.emit (events.js:314:20)",
"    at processPromiseRejections (internal/process/promises.js:209:33)",
"    at processTicksAndRejections (internal/process/task_queues.js:98:32)"
]
}

@lindsaylevine
Copy link

@MejanH ok WEW finally got this deployed and diagnosed! so here's what's going on:

because these pages are trying to use ISR (see revalidate flag in your getStaticProps functions), our plugin turns these pages into netlify functions. from within the function['s node env], the files (aka the markdown) you're trying to read "from disk" are not bundled with those functions, so it's failing to find them and erroring out.

this is something we can/should address, but for now, if you want this to Just Work, don't use ISR/revalidate. it will work as a plain SSG'd/prerendered page.

thanks for opening that separate repo for me and let me know if you have any other questions!! i'm going to change the title if that's cool to be a little more specific to the issue :)

@lindsaylevine lindsaylevine changed the title Not working with Markdown blog Markdown files cannot be found in netlify function environment Dec 15, 2020
@MejanH
Copy link
Author

MejanH commented Dec 15, 2020

Thanks for the help. Will wait for this feature.

@lindsaylevine lindsaylevine transferred this issue from netlify/next-runtime Dec 21, 2020
@lindsaylevine
Copy link

@MejanH hey! we pushed up a temporary solution for this supported in the plugin and wanted to let you know since you were the very first to surface this issue! i will note that this is a temporary, unstable solution. we're also working on a greater, platform-wide solution for this type of file inclusion. if you'd like access to the short term next-specific plugin solution, feel free to email me lindsay@netlify.com or dm on twitter @levlinds. otherwise, i'll leave this issue open until the longer term solution rolls out! <3

@joelvarty
Copy link

@MejanH hey! we pushed up a temporary solution for this supported in the plugin and wanted to let you know since you were the very first to surface this issue! i will note that this is a temporary, unstable solution. we're also working on a greater, platform-wide solution for this type of file inclusion. if you'd like access to the short term next-specific plugin solution, feel free to email me lindsay@netlify.com or dm on twitter @levlinds. otherwise, i'll leave this issue open until the longer term solution rolls out! <3

Reaching out to you on twitter for this magical temporary solution!

@lindsaylevine
Copy link

the platform solution is finally out!!! https://docs.netlify.com/configure-builds/file-based-configuration/#functions

please go ahead and give it a shot!

@coreyaus
Copy link

Thanks for the good news @lindsaylevine! Is this also intended to work with Server-side Rendered pages using getServerSideProps? I ran a quick test but couldn't get it to work. If there are any examples available of this in use with SSR then that would be amazing to see (understand if that's not possible though!)

@lindsaylevine
Copy link

@coreyaus hey! yes it should work with getServerSideProps! mind sharing how you're reading these files in your page, where the files are in your project, and how you configured your netlify.toml? that'd help me figure out why it's not working for you! :)

@coreyaus
Copy link

coreyaus commented May 20, 2021

Thanks for the prompt response @lindsaylevine!

I've got this up and running now. Essentially I found that process.cwd() didn't work on Netlify (but did work in local development, and on Vercel. For more context, the previous version of getServerSideProps I was using accessed the filesystem in a manner something like the below (I have inlined a helper function here for ease of reference):

import matter from "gray-matter";

export const getServerSideProps = async ({ params }) => {
  const fs = require("fs");
  const path = require("path");
  const directoryPath = path.join(process.cwd(), "_posts");
  const pageSlugs = fs.readdirSync(directoryPath);
  const allPosts = pageSlugs.map((slug) => {
    const fullPath = path.join(directoryPath, slug);
    const fileContents = fs.readFileSync(fullPath, "utf8");
    const document = matter(fileContents);
    return { slug, ...document };
  });

  return {
    props: { ...params, allPosts },
  };
};

The revised version that does work across localhost, Netlify and Vercel borrows heavily from this repository: https://github.com/cassidoo/next-netlify-blog-starter and looks more like the below, where the getPosts method is now pretty much identical to this: https://github.com/cassidoo/next-netlify-blog-starter/blob/master/utils/getPosts.js

import matter from "gray-matter";
import getPosts from "path/to/utils/file";

export const getServerSideProps = async ({ params }) => {
  const allPosts = ((context) => {
    return getPosts(context);
  })(require.context("../../posts", true, /\.md$/));

  return {
    props: { ...params, allPosts },
  };
};

For that code to work you also need to run yarn add raw-loader and include this setting in your next.config.js:

config.module.rules.push({
    test: /\.md$/,
    use: "raw-loader",
});

It would be handy to have process.cwd() set to the project root like it is in localhost and Vercel to avoid always having to include a hard-coded string reference like "../../posts" or "../posts" or whatever, but not a big deal. Always open to ideas if people know of ways to make that reference more dynamic while staying on Netlify. At the moment process.cwd() === '/var/task' in the Netlify functions created for Server-Side Rendered (SSR) pages

Anyway, mostly just sharing in case these notes help anyone else (and also as a plug for all the excellent Netlify start repos that exist like the one linked in this comment!) 😄

PS: might also be worth posting the good news about the latest platform solution for this problem here too: https://answers.netlify.com/t/how-to-access-files-from-nextjs-api-function/35347/2

@lindsaylevine
Copy link

@coreyaus hey corey! per @eduardoboucas aka the engineer behind included_files, this is something the team is working on (the process.cwd() stuff)!

thanks so much for sharing your notes regardless, we appreciate it and i'm sure other users will too!!

i also left a comment on that community thread 👍

@juanpprieto
Copy link

juanpprieto commented Jun 3, 2021

@lindsaylevine I tried, 3.4.0 included_files for reading .json but unfortunately it does not work when in preview mode. I get /var/task/public/json/xyz.json ... not found. (also tried reading from the path /var/task/json/xyz.json without any luck). Local netlify build` in both production and preview work just fine. :(

My workaround has been using fetch to get these json files but I'd love to remove this logic in the hope that it would speed up our preview mode experience.

How is process.cwd support coming along?

Do you have any idea when this will be generally available and where would be the best way to find out when it's ready? - I ask specially as this seems as "small" feature in netlify's nextjs support scope

thanks in advance!

@lindsaylevine
Copy link

lindsaylevine commented Jun 3, 2021

@juanpprieto hey there! my apologies for not circling back here!!! my teammate @eduardoboucas just recently shed some light on that question here. per his most recent comment, it's something we'll be rolling out soon. you can subscribe to #359 to see when eduardo gives that release update!

@eduardoboucas
Copy link
Member

@juanpprieto the fix that @lindsaylevine mentioned has now been rolled out. Can you please check if it works for you? Thanks!

@juanpprieto
Copy link

juanpprieto commented Jun 21, 2021

Thanks @eduardoboucas.

Unfortunately, I haven't had any luck. I just tried this on "@netlify/plugin-nextjs": "3.4.2", but I'm either not accessing .jsons correctly or it's not working.

netlify.toml

[functions]
  node_bundler = "esbuild"
  included_files = ["json/collections/*.json"]

.json & getStaticProps

My .jsons live on /public/collections/*.json. I'm then trying to read the .json on the collection's getStaticProps with something like (pseudo-code):

const collection = await fs.readFile(path.join(process.cwd(), `/json/collections/${params.handle}.json`), 'utf8')

ENOENT: no such file or directory, open '/var/task/json/collections/tires.json'

Console logging process.cwd() still returns /var/task, so I get nothing back

Notes

I inspected the /out_functions/next_collections_handle.js and I don't see any references to these json files.

Any idea what I may be doing wrong?

P.S reading from the filesystem works fine on every context except (isPreview === true && NODE_ENV === 'production')

@eduardoboucas
Copy link
Member

Thanks @eduardoboucas.

Unfortunately, I haven't had any luck. I just tried this on "@netlify/plugin-nextjs": "3.4.2", but I'm either not accessing .jsons correctly or it's not working.

netlify.toml

[functions]
  node_bundler = "esbuild"
  included_files = ["json/collections/*.json"]

.json & getStaticProps

My .jsons live on /public/collections/*.json. I'm then trying to read the .json on the collection's getStaticProps with something like (pseudo-code):

const collection = await fs.readFile(path.join(process.cwd(), `/json/collections/${params.handle}.json`), 'utf8')

ENOENT: no such file or directory, open '/var/task/json/collections/tires.json'

Console logging process.cwd() still returns /var/task, so I get nothing back

Notes

I inspected the /out_functions/next_collections_handle.js and I don't see any references to these json files.

Any idea what I may be doing wrong?

P.S reading from the filesystem works fine on every context except (isPreview === true && NODE_ENV === 'production')

Hi @juanpprieto.

The path should be relative to the project root, so if the files live under /public, I believe the config should be:

[functions]
  node_bundler = "esbuild"
  included_files = ["public/json/collections/*.json"]

And to read them you can try:

const collection = await fs.readFile(path.resolve(`public/json/collections/${params.handle}.json`), 'utf8')

Let me know if this helps.

@juanpprieto
Copy link

Game changer! 🔥🔥🔥

Thanks! @eduardoboucas

foxprogs added a commit to texttree/opencomponents.io that referenced this issue Dec 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority: high type: bug code to address defects in shipped code
Projects
None yet
Development

No branches or pull requests

6 participants