Skip to content

feat: aem content#2689

Merged
kptdobe merged 43 commits intomainfrom
aem-content
Apr 16, 2026
Merged

feat: aem content#2689
kptdobe merged 43 commits intomainfrom
aem-content

Conversation

@kptdobe
Copy link
Copy Markdown
Contributor

@kptdobe kptdobe commented Mar 19, 2026

Allows to locally checkout the project content, make modifications, push back and serve locally. Useful for bulk editing.

Command Description
aem content clone Clone da.live content locally into content/
aem content status Show locally added, modified, and deleted content files
aem content diff Show diff between local and remote content
aem content merge Merge remote content into local files
aem content add Stage changes in content/ (like git add)
aem content commit Commit staged changes in content/ (like git commit)
aem content push Push committed content/ changes to da.live (use content add & content commit first)

if content folder is present, aem up serves content first from here (and injects head.html and simulates metadata)

@kptdobe kptdobe changed the title Aem content feat: aem content Mar 19, 2026
Comment thread src/server/HelixServer.js Fixed
@github-actions
Copy link
Copy Markdown

This PR will trigger a minor release when merged.

Comment thread src/content/clone.cmd.js Fixed
Comment thread src/content/clone.cmd.js Outdated
Comment thread src/server/content-metadata-html.js Outdated
Comment thread src/content/content-metadata-html.js
Comment thread src/server/content-metadata-html.js Outdated
Copy link
Copy Markdown
Contributor

@tripodsan tripodsan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@davidnuescheler IIUC, you try to re-create the same that is done in the pipeline, by transforming metadata etc... why don't we use the html2md and html-pipline if really this is what we want to do here.

@davidnuescheler
Copy link
Copy Markdown
Contributor

@tripodsan i think this is a rough approximation anyway, as the source is a kind of messy ingested HTML that is very different from the clean .md generated html in the pipeline.. i think simplicity and approximation is more important here, than precision.

@davidnuescheler davidnuescheler marked this pull request as ready for review March 27, 2026 17:34
Comment thread test/content-metadata-html.test.js Fixed
Comment thread src/server/MetadataSheetSupport.js Fixed
@davidnuescheler
Copy link
Copy Markdown
Contributor

Summary

Adds aem content workflows so developers can clone da.live project content into a local content/ tree, edit it offline, inspect changes (status, diff), merge with remote when both sides changed, stage/commit (git-like add / commit with message), and push updates back to da.live. When content/ exists, aem up prefers local files for matching paths and wraps body-only HTML with head.html plus metadata handling for a realistic local preview.

Use cases include bulk editing, scripted workflows, and working without the authoring UI.


Commands

Command Purpose
aem content clone Download da.live sources under ./content/, initialize a git repo, baseline commit, write .da-config.json (org, repo, root path), and record sync state. Supports --path, --force, --yes for large clones.
aem content status Show added / modified / deleted files under content/.
aem content diff Compare local working tree to da.live (optional --path).
aem content merge Merge remote content into local files when both changed (optional path; merges all conflicted paths if omitted).
aem content add Stage paths (like git add).
aem content commit Commit staged changes with a message (like git commit).
aem content push Upload commits to da.live; supports --dry-run, --path, --force for conflict handling.

Authentication: IMS implicit OAuth via browser; token may be passed with --token (e.g. CI). Tokens are stored under ~/.aem/da-token.json (with expiry). After a successful browser login, the callback redirects to https://tools.aem.live/cli/logged-in.


aem up and local preview

  • If content/ exists and the dev server runs in proxy mode, requests try content/ first; missing assets fall through to the preview proxy.
  • HTML under content/ is treated as body fragments where applicable: the server injects head.html, may merge /metadata.json row matches for extra <meta> tags, and runs transformContentMetadataHtml to turn da.live-style <div class="metadata"> blocks into <meta> tags and strip them from the body—lightweight approximation of production pipeline behavior, not a full html-pipeline/html2md pass (intentional tradeoff for dev speed).

Implementation notes

  • da.live API client (DaClient): list (with continuation), get/put/delete source, bounded concurrency for recursive listing and I/O.
  • Shared helpers (content-shared.js): CONTENT_DIR (content), processQueue (bounded parallel I/O for clone/push), path normalization, clone size threshold.
  • Git in content/ via isomorphic-git; sync ref / helpers in content-git.js.
  • Metadata sheet integration where applicable (MetadataSheetSupport, path matching) for sheet-driven metas alongside page metadata.

Review and tooling

  • CodeQL / Copilot autofix–style items addressed where applicable (e.g. safer URL assertions in tests, guard cleanup in MetadataSheetSupport).
  • Code review feedback: e.g. use hast-util-select instead of ad-hoc HAST walks for first h1 / p / img[src]; rename generic truncateWithEllipsis; bounded concurrency named processQueue locally (aligned with review naming; @adobe/helix-shared-utils does not export processQueue in the published utils entry, so a small local helper avoids a dependency-only pull).

Testing

  • Run npm test (and lint as in CI). Content-related tests live under test/content/; server/metadata tests cover content-metadata-html and related paths.

BREAKING CHANGE: none expected for existing CLI users who do not use content/; new behavior activates when the content subcommand is used or the content/ directory is present for aem up.

@kptdobe
Copy link
Copy Markdown
Contributor Author

kptdobe commented Apr 8, 2026

ok, went through all feedback. Also saw we've merge 4ed1f4b yesterday which has some overlaps...

@trieloff
Copy link
Copy Markdown
Contributor

@shsteimer fyi, we'll have to update a bunch of skills for this.

Comment thread src/content/da-auth.js Outdated
if (token) {
window.location.href = loggedInUrl;
} else {
document.body.innerHTML = '<h2>Login failed.</h2><p>' + (error || 'Unknown error') + '</p>';
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use error without escaping in innerHTML

Comment thread src/content/da-auth.js Outdated
const loggedInUrl = 'https://tools.aem.live/cli/logged-in';
fetch(dest)
.then(() => {
if (token) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is strange... why do you fetch if the token is missing?

Comment thread src/content/da-auth.js Outdated
Comment on lines +53 to +60
if (!await GitUtils.isIgnored(projectDir, DA_TOKEN_FILE)) {
await fs.appendFile(
path.join(projectDir, '.gitignore'),
`${os.EOL}${DA_TOKEN_FILE}${os.EOL}`,
'utf8',
);
}
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why doesn't this use ensureGitIgnored( ) from content-git.js ?

* @param {string} input
* @returns {string}
*/
export function normalizeDaPath(input) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks over complicated and doesn't just normalize, but also validate. maybe add at least throws jsdoc

Comment thread src/content/da-api.js Outdated
continuation = next;
}

throw new Error(`List pagination for ${daPath} exceeded ${LIST_MAX_PAGES} pages.`);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not very user friendly... shouldn't it just return the amount of pages fetched?

Comment thread src/content/push.cmd.js Outdated
return this;
}

async run() {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this is a very long function. would be nicer to split....

Comment thread src/server/content-metadata-html.js Outdated
if (node.type === 'text') {
return node.value;
}
if ('children' in node && Array.isArray(node.children)) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't this just Array.isArray(node.children ?

> node = {}
> Array.isArray(node.children)
false

* OF ANY KIND, either express or implied. See the License for the specific language
* governing permissions and limitations under the License.
*/

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thnk this file also belongs in the src/content directory

* @param {Array<{ pattern: string, regex: RegExp, row: object }>} compiled
* @returns {Record<string, string> | null}
*/
export function mergeMetadataSheetRows(normalizedPath, compiled) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why don't you use the ModifiersConfig from helix-shared? it seems the entire metadata logic is re-implemented here

kptdobe and others added 10 commits April 16, 2026 15:30
The saveDaTokenToFile function now uses ensureGitIgnored from
content-git.js instead of GitUtils.isIgnored + fs.appendFile.
Update the test to mock content-git.ensureGitIgnored and assert
it receives the expected token file path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@kptdobe kptdobe merged commit 5524b6d into main Apr 16, 2026
9 checks passed
@kptdobe kptdobe deleted the aem-content branch April 16, 2026 14:33
adobe-bot pushed a commit that referenced this pull request Apr 16, 2026
# [16.18.0](v16.17.1...v16.18.0) (2026-04-16)

### Features

* aem content ([#2689](#2689)) ([5524b6d](5524b6d))
@adobe-bot
Copy link
Copy Markdown
Collaborator

🎉 This PR is included in version 16.18.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants