Tools: building website abends

Since the GSoC ideas were updated I thought I'd build the web site.

Previously I'd just run node build-website.js in packages/tools, but this deletes all files in ../../docs/api/references/plugin_api/. Hmmm, very weird. Ok, different approach then.

From the root dir I tried to run npm run buildWebsite. Unfortunately it abends:

$ npm run buildWebsite

> buildWebsite
> npm run buildApiDoc && node ./packages/tools/build-website.js && npm run buildPluginDoc && npm run buildSettingJsonSchema


> buildApiDoc
> npm start --prefix=packages/app-cli -- apidoc ../../readme/api/references/rest_api.md


> joplin@1.8.0 start
> gulp build -L && node "build/main.js" --stack-trace-enabled --log-level debug --env dev "apidoc" "../../readme/api/references/rest_api.md"


> buildPluginDoc
> typedoc --name 'Joplin Plugin API Documentation' --mode file -theme './Assets/PluginDocTheme/' --readme './Assets/PluginDocTheme/index.md' --excludeNotExported --excludeExternals --excludePrivate --excludeProtected --out docs/api/references/plugin_api packages/lib/services/plugins/api/


Using TypeScript 4.0.5 from /Users/tessus/data/work/joplin/node_modules/typescript/lib
Rendering [========================================] 100%

Documentation generated at /Users/tessus/data/work/joplin/docs/api/references/plugin_api


> buildSettingJsonSchema
> npm start --prefix=packages/app-cli -- settingschema ../../docs/schema/settings.json


> joplin@1.8.0 start
> gulp build -L && node "build/main.js" --stack-trace-enabled --log-level debug --env dev "settingschema" "../../docs/schema/settings.json"

Error: No such command: settingschema
    at Application.findCommandByName (/Users/tessus/data/work/joplin/packages/app-cli/build/app.js:255:15)
    at Application.execCommand (/Users/tessus/data/work/joplin/packages/app-cli/build/app.js:299:30)
    at Application.start (/Users/tessus/data/work/joplin/packages/app-cli/build/app.js:406:16)
    at processTicksAndRejections (internal/process/task_queues.js:93:5) {
  type: 'notFound'
}
npm ERR! code 1
npm ERR! path /Users/tessus/data/work/joplin/packages/app-cli
npm ERR! command failed
npm ERR! command sh -c gulp build -L && node "build/main.js" --stack-trace-enabled --log-level debug --env dev "settingschema" "../../docs/schema/settings.json"

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/tessus/.npm/_logs/2021-02-18T23_37_42_607Z-debug.log
npm ERR! code 1
npm ERR! path /Users/tessus/data/work/joplin
npm ERR! command failed
npm ERR! command sh -c npm start --prefix=packages/app-cli -- settingschema ../../docs/schema/settings.json

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/tessus/.npm/_logs/2021-02-18T23_37_42_695Z-debug.log
npm ERR! code 1
npm ERR! path /Users/tessus/data/work/joplin
npm ERR! command failed
npm ERR! command sh -c npm run buildApiDoc && node ./packages/tools/build-website.js && npm run buildPluginDoc && npm run buildSettingJsonSchema

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/tessus/.npm/_logs/2021-02-18T23_37_42_735Z-debug.log

@laurent I saw that you updated the web site. Which command did you run? Is it possible that I'm missing a dependency even though I ran npm i in the root dir?

It should be npm run buildDoc but I'm not sure in your case why it's not finding the settingschema command. It should have been built with npm i

I've just pulled the latest changes from dev and ran npm i. Unfortunately it also abends:

$ npm i

> postinstall
> npm run bootstrap --no-ci && npm run build


> bootstrap
> lerna bootstrap --no-ci

lerna notice cli v3.22.1
lerna info versioning independent
lerna info Bootstrapping 15 packages
lerna info Installing external dependencies
lerna info Symlinking packages and binaries
lerna info lifecycle @joplin/app-clipper@1.0.8~postinstall: @joplin/app-clipper@1.0.8
lerna info lifecycle @joplin/app-desktop@1.8.0~postinstall: @joplin/app-desktop@1.8.0
lerna info lifecycle @joplin/app-mobile@0.8.9~postinstall: @joplin/app-mobile@0.8.9

> @joplin/app-clipper@1.0.8 postinstall /Users/tessus/data/work/joplin/packages/app-clipper
> cd popup && npm install


> joplin-webclipper-popup@0.1.0 postinstall
> node postinstall.js && npm run build


> joplin-webclipper-popup@0.1.0 build
> node scripts/build.js SKIP_PREFLIGHT_CHECK

Creating an optimized production build...
Browserslist: caniuse-lite is outdated. Please run the following command: `npm update`
Compiled successfully.

File sizes after gzip:

  48.12 KB  build/static/js/2.chunk.js
  7.52 KB   build/static/js/main.chunk.js
  775 B     build/static/js/runtime-main.js
  746 B     build/static/css/main.chunk.css

The project was built assuming it is hosted at ./.
You can control this with the homepage field in your package.json.

The build folder is ready to be deployed.

Find out more about deployment here:

  bit.ly/CRA-deploy


up to date, audited 2128 packages in 9s

27 vulnerabilities (13 low, 1 moderate, 13 high)

To address issues that do not require attention, run:
  npm audit fix

To address all issues (including breaking changes), run:
  npm audit fix --force

Run `npm audit` for details.
lerna info lifecycle @joplin/app-clipper@1.0.8~postinstall: Failed to exec postinstall script

> @joplin/app-desktop@1.8.0 postinstall /Users/tessus/data/work/joplin/packages/app-desktop
> npm run build

npm ERR! code 1
npm ERR! path /Users/tessus/data/work/joplin
npm ERR! command failed
npm ERR! command sh -c lerna bootstrap --no-ci

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/tessus/.npm/_logs/2021-02-19T23_24_00_380Z-debug.log
npm ERR! code 1
npm ERR! path /Users/tessus/data/work/joplin
npm ERR! command failed
npm ERR! command sh -c npm run bootstrap --no-ci && npm run build

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/tessus/.npm/_logs/2021-02-19T23_24_00_469Z-debug.log

> @joplin/app-desktop@1.8.0 build
> gulp build

[tessus@epsilon3 1 ~/work/joplin dev {upstream/dev}|✚6⚑1]$ /Users/tessus/data/work/joplin
[18:24:01] Using gulpfile ~/data/work/joplin/packages/app-desktop/gulpfile.js
[18:24:01] Starting 'build'...
[18:24:01] Starting 'compileScripts'...
[18:24:01] Starting 'compilePackageInfo'...
[18:24:01] Starting 'copyPluginAssets'...
[18:24:01] Starting 'copyTinyMceLangs'...
[18:24:01] Starting 'updateIgnoredTypeScriptBuild'...
Copying to /Users/tessus/data/work/joplin/packages/app-desktop/tools/../gui/note-viewer/pluginAssets
Copying /Users/tessus/data/work/joplin/packages/app-desktop/tools/../../../Assets/TinyMCE/langs => /Users/tessus/data/work/joplin/packages/app-desktop/tools/../node_modules/tinymce/langs
[18:24:03] Finished 'compileScripts' after 2.36 s
[18:24:03] Finished 'compilePackageInfo' after 2.36 s
[18:24:03] Finished 'updateIgnoredTypeScriptBuild' after 2.37 s
Copying to /Users/tessus/data/work/joplin/packages/app-desktop/tools/../pluginAssets
[18:24:03] Finished 'copyTinyMceLangs' after 2.45 s
[18:24:03] Finished 'copyPluginAssets' after 2.49 s
[18:24:03] Finished 'build' after 2.49 s
Here are the complete log files
$ cat /Users/tessus/.npm/_logs/2021-02-19T23_24_00_380Z-debug.log
0 verbose cli [
0 verbose cli   '/usr/local/lib/nodejs/node-v14.15.0/bin/node',
0 verbose cli   '/Users/tessus/.npm-global/bin/npm',
0 verbose cli   'run',
0 verbose cli   'bootstrap',
0 verbose cli   '--no-ci'
0 verbose cli ]
1 info using npm@7.5.4
2 info using node@v14.15.0
3 timing config:load:defaults Completed in 1ms
4 timing config:load:file:/Users/tessus/.npm-global/lib/node_modules/npm/npmrc Completed in 1ms
5 timing config:load:builtin Completed in 1ms
6 timing config:load:cli Completed in 1ms
7 timing config:load:env Completed in 0ms
8 timing config:load:file:/Users/tessus/data/work/joplin/.npmrc Completed in 0ms
9 timing config:load:project Completed in 1ms
10 timing config:load:file:/Users/tessus/.npmrc Completed in 0ms
11 timing config:load:user Completed in 0ms
12 timing config:load:file:/Users/tessus/.npm-global/etc/npmrc Completed in 0ms
13 timing config:load:global Completed in 0ms
14 timing config:load:cafile Completed in 1ms
15 timing config:load:validate Completed in 0ms
16 timing config:load:setUserAgent Completed in 0ms
17 timing config:load:setEnvs Completed in 1ms
18 timing config:load Completed in 6ms
19 verbose npm-session 102e5042a11ae71c
20 timing npm:load Completed in 12ms
21 timing command:run-script Completed in 119919ms
22 verbose stack Error: command failed
22 verbose stack     at ChildProcess.<anonymous> (/Users/tessus/.npm-global/lib/node_modules/npm/node_modules/@npmcli/promise-spawn/index.js:64:27)
22 verbose stack     at ChildProcess.emit (events.js:315:20)
22 verbose stack     at maybeClose (internal/child_process.js:1048:16)
22 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5)
23 verbose cwd /Users/tessus/data/work/joplin
24 verbose Darwin 18.7.0
25 verbose argv "/usr/local/lib/nodejs/node-v14.15.0/bin/node" "/Users/tessus/.npm-global/bin/npm" "run" "bootstrap" "--no-ci"
26 verbose node v14.15.0
27 verbose npm  v7.5.4
28 error code 1
29 error path /Users/tessus/data/work/joplin
30 error command failed
31 error command sh -c lerna bootstrap --no-ci
32 verbose exit 1
$ cat /Users/tessus/.npm/_logs/2021-02-19T23_24_00_469Z-debug.log
0 verbose cli [
0 verbose cli   '/usr/local/lib/nodejs/node-v14.15.0/bin/node',
0 verbose cli   '/Users/tessus/.npm-global/bin/npm',
0 verbose cli   'i'
0 verbose cli ]
1 info using npm@7.5.4
2 info using node@v14.15.0
3 timing config:load:defaults Completed in 1ms
4 timing config:load:file:/Users/tessus/.npm-global/lib/node_modules/npm/npmrc Completed in 1ms
5 timing config:load:builtin Completed in 1ms
6 timing config:load:cli Completed in 0ms
7 timing config:load:env Completed in 1ms
8 timing config:load:file:/Users/tessus/data/work/joplin/.npmrc Completed in 0ms
9 timing config:load:project Completed in 0ms
10 timing config:load:file:/Users/tessus/.npmrc Completed in 1ms
11 timing config:load:user Completed in 1ms
12 timing config:load:file:/Users/tessus/.npm-global/etc/npmrc Completed in 0ms
13 timing config:load:global Completed in 0ms
14 timing config:load:cafile Completed in 0ms
15 timing config:load:validate Completed in 1ms
16 timing config:load:setUserAgent Completed in 0ms
17 timing config:load:setEnvs Completed in 0ms
18 timing config:load Completed in 5ms
19 verbose npm-session e7bedea4b2e34665
20 timing npm:load Completed in 12ms
21 timing arborist:ctor Completed in 1ms
22 timing idealTree:init Completed in 382ms
23 timing idealTree:userRequests Completed in 0ms
24 silly idealTree buildDeps
25 timing idealTree:#root Completed in 0ms
26 timing idealTree:buildDeps Completed in 1ms
27 timing idealTree:fixDepFlags Completed in 0ms
28 timing idealTree Completed in 384ms
29 timing arborist:ctor Completed in 0ms
30 timing reify:loadTrees Completed in 606ms
31 timing reify:diffTrees Completed in 13ms
32 silly reify moves {}
33 timing reify:retireShallow Completed in 0ms
34 timing reify:createSparse Completed in 0ms
35 timing reify:loadBundles Completed in 0ms
36 timing reify:unpack Completed in 7ms
37 timing reify:unretire Completed in 0ms
38 timing build:queue Completed in 0ms
39 timing build:deps Completed in 0ms
40 timing build Completed in 0ms
41 timing reify:build Completed in 0ms
42 timing reify:trash Completed in 1ms
43 http fetch POST 200 https://registry.npmjs.org/-/npm/v1/security/advisories/bulk 399ms
44 timing auditReport:getReport Completed in 402ms
45 timing metavuln:cache:get:security-advisory:ini:Ycq69/gaCzr/G/fbN43rDLIsHS7IdZHzW5bh0ju4uwHwvENyyzefC75bnxzsKQ1M+jh/75af+MnibiJLXd61Iw== Completed in 9ms
46 timing reify:save Completed in 512ms
47 http fetch GET 304 https://registry.npmjs.org/ini 186ms (from cache)
48 timing metavuln:packument:ini Completed in 188ms
49 timing metavuln:load:security-advisory:ini:1589 Completed in 0ms
50 timing metavuln:calculate:security-advisory:ini:1589 Completed in 188ms
51 timing metavuln:cache:get:security-advisory:global-prefix:2Y1tVIAOCQDQ7wuh0WDPOPCyHKpY0ClBm+v4U6dukj413sf6DLnn0+K5kGceXZ1HOmT7cfj8aBWZWnFE2DXNkg== Completed in 13ms
52 timing metavuln:cache:get:security-advisory:rc:/2MFlZ/T1CPQQswjJhNhhlG4q5EG45VjjuYqMInN8sTOeP2ga8AcQhNm9qWbv2d5ZF/9iKyTNk1faKbtVHHNYg== Completed in 12ms
53 timing metavuln:cache:get:security-advisory:gitconfiglocal:7khazJ4vKK2hW1ncyF57L8iczboD0gJUlqiFJfMYVCms/v3qOfDnlKhMpXjswhm0V0Ab5u/X17WxkglTQNO3FA== Completed in 13ms
54 timing metavuln:cache:get:security-advisory:config-chain:/wNYO5/PZ/TMyocbR/sMEk6kzCih8ZU0AJ1XlupzixBTzLGJD8M0PHlE/J6IADbzOMfA50f9F8q/TNbssGAIZw== Completed in 15ms
55 http fetch GET 304 https://registry.npmjs.org/config-chain 57ms (from cache)
56 timing metavuln:packument:config-chain Completed in 58ms
57 timing metavuln:load:security-advisory:config-chain:Ycq69/gaCzr/G/fbN43rDLIsHS7IdZHzW5bh0ju4uwHwvENyyzefC75bnxzsKQ1M+jh/75af+MnibiJLXd61Iw== Completed in 1ms
58 timing metavuln:calculate:security-advisory:config-chain:Ycq69/gaCzr/G/fbN43rDLIsHS7IdZHzW5bh0ju4uwHwvENyyzefC75bnxzsKQ1M+jh/75af+MnibiJLXd61Iw== Completed in 59ms
59 http fetch GET 304 https://registry.npmjs.org/gitconfiglocal 131ms (from cache)
60 timing metavuln:packument:gitconfiglocal Completed in 131ms
61 timing metavuln:load:security-advisory:gitconfiglocal:Ycq69/gaCzr/G/fbN43rDLIsHS7IdZHzW5bh0ju4uwHwvENyyzefC75bnxzsKQ1M+jh/75af+MnibiJLXd61Iw== Completed in 0ms
62 timing metavuln:calculate:security-advisory:gitconfiglocal:Ycq69/gaCzr/G/fbN43rDLIsHS7IdZHzW5bh0ju4uwHwvENyyzefC75bnxzsKQ1M+jh/75af+MnibiJLXd61Iw== Completed in 131ms
63 http fetch GET 304 https://registry.npmjs.org/global-prefix 139ms (from cache)
64 timing metavuln:packument:global-prefix Completed in 140ms
65 timing metavuln:load:security-advisory:global-prefix:Ycq69/gaCzr/G/fbN43rDLIsHS7IdZHzW5bh0ju4uwHwvENyyzefC75bnxzsKQ1M+jh/75af+MnibiJLXd61Iw== Completed in 0ms
66 timing metavuln:calculate:security-advisory:global-prefix:Ycq69/gaCzr/G/fbN43rDLIsHS7IdZHzW5bh0ju4uwHwvENyyzefC75bnxzsKQ1M+jh/75af+MnibiJLXd61Iw== Completed in 141ms
67 http fetch GET 304 https://registry.npmjs.org/rc 141ms (from cache)
68 timing metavuln:packument:rc Completed in 141ms
69 timing metavuln:load:security-advisory:rc:Ycq69/gaCzr/G/fbN43rDLIsHS7IdZHzW5bh0ju4uwHwvENyyzefC75bnxzsKQ1M+jh/75af+MnibiJLXd61Iw== Completed in 0ms
70 timing metavuln:calculate:security-advisory:rc:Ycq69/gaCzr/G/fbN43rDLIsHS7IdZHzW5bh0ju4uwHwvENyyzefC75bnxzsKQ1M+jh/75af+MnibiJLXd61Iw== Completed in 141ms
71 timing auditReport:init Completed in 332ms
72 timing reify:audit Completed in 735ms
73 timing reify Completed in 1360ms
74 timing command:install Completed in 121596ms
75 verbose stack Error: command failed
75 verbose stack     at ChildProcess.<anonymous> (/Users/tessus/.npm-global/lib/node_modules/npm/node_modules/@npmcli/promise-spawn/index.js:64:27)
75 verbose stack     at ChildProcess.emit (events.js:315:20)
75 verbose stack     at maybeClose (internal/child_process.js:1048:16)
75 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5)
76 verbose cwd /Users/tessus/data/work/joplin
77 verbose Darwin 18.7.0
78 verbose argv "/usr/local/lib/nodejs/node-v14.15.0/bin/node" "/Users/tessus/.npm-global/bin/npm" "i"
79 verbose node v14.15.0
80 verbose npm  v7.5.4
81 error code 1
82 error path /Users/tessus/data/work/joplin
83 error command failed
84 error command sh -c npm run bootstrap --no-ci && npm run build
85 verbose exit 1

But I think the issue is:

22 verbose stack Error: command failed
22 verbose stack     at ChildProcess.<anonymous> (/Users/tessus/.npm-global/lib/node_modules/npm/node_modules/@npmcli/promise-spawn/index.js:64:27)
22 verbose stack     at ChildProcess.emit (events.js:315:20)
22 verbose stack     at maybeClose (internal/child_process.js:1048:16)
22 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5)

P.S.: I also tried running npm run clean before npm i, but to no avail.

Found the problem:

I had to downgrade npm from 7.5.4 to 6.14.11.