Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ build-api-reference:
cd website && poetry run ./build_api_reference.sh

build-docs:
cd website && corepack enable && yarn && yarn build
cd website && corepack enable && yarn && poetry run yarn build

run-docs: build-api-reference
cd website && corepack enable && yarn && yarn start
cd website && corepack enable && yarn && poetry run yarn start
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the plugin now runs pydoc-markdown (and other python tooling) automatically, we need to run the Docusaurus build with the correct venv using poetry run (hopefully this is how it works).

8 changes: 0 additions & 8 deletions website/build_api_reference.sh
Original file line number Diff line number Diff line change
@@ -1,12 +1,4 @@
#!/bin/bash

# Create docspec dump of this package's source code through pydoc-markdown
python ./pydoc-markdown/generate_ast.py > docspec-dump.jsonl

rm -rf "${apify_shared_tempdir}"

# Generate import shortcuts from the modules
python generate_module_shortcuts.py

# Transform the docpec dumps into Typedoc-compatible docs tree
node transformDocs.js
32 changes: 28 additions & 4 deletions website/docusaurus.config.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,27 @@
/* eslint-disable global-require,import/no-extraneous-dependencies */
/* eslint-disable global-require */
const path = require('path');

const { externalLinkProcessor } = require('./tools/utils/externalLink');
const { groupSort } = require('./transformDocs');

const GROUP_ORDER = [
'Classes',
'Abstract classes',
'Data structures',
'Errors',
'Functions',
'Constructors',
'Methods',
'Properties',
'Constants',
'Enumeration Members',
];

const groupSort = (g1, g2) => {
if (GROUP_ORDER.includes(g1) && GROUP_ORDER.includes(g2)) {
return GROUP_ORDER.indexOf(g1) - GROUP_ORDER.indexOf(g2);
}
return g1.localeCompare(g2);
};

/** @type {Partial<import('@docusaurus/types').DocusaurusConfig>} */
module.exports = {
Expand All @@ -14,6 +35,7 @@ module.exports = {
'/python/js/custom.js',
'/crawlee-python/js/custom.js',
],
githubHost: 'github.com',
headTags: [
// Intercom messenger
{
Expand Down Expand Up @@ -78,9 +100,11 @@ module.exports = {
excludeExternals: false,
},
sortSidebar: groupSort,
pathToCurrentVersionTypedocJSON: `${__dirname}/api-typedoc-generated.json`,
routeBasePath: 'api',
python: true,
pythonOptions: {
pythonModulePath: path.join(__dirname, '../src/crawlee'),
moduleShortcutsPath: path.join(__dirname, 'module_shortcuts.json'),
},
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new unified API docs generating pipeline - we don't need to generate the json ourselves, just supply the path to the project (and the module_shortcuts.json file, which we might want to remove / integrate later, further simplifying the interface).

},
],
// [
Expand Down
2 changes: 1 addition & 1 deletion website/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
"typescript": "5.6.3"
},
"dependencies": {
"@apify/docusaurus-plugin-typedoc-api": "^4.2.7",
"@apify/docusaurus-plugin-typedoc-api": "^4.3.0",
"@apify/utilities": "^2.8.0",
"@docusaurus/core": "^3.5.2",
"@docusaurus/mdx-loader": "^3.5.2",
Expand Down
82 changes: 0 additions & 82 deletions website/parse_types.py

This file was deleted.

14 changes: 0 additions & 14 deletions website/pydoc-markdown.yml

This file was deleted.

Loading
Loading