Skip to content

fix(amazonq): fix processing empty unsupported workspace file #2017

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Aug 1, 2025
Merged

fix(amazonq): fix processing empty unsupported workspace file #2017

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
bbfeb9f
fix(amazonq): fix processing empty unsupported workspace file
liumofei-amazon Jul 31, 2025
612766f
Merge branch 'main' into classpathfix
liumofei-amazon Aug 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 30 additions & 7 deletions server/aws-lsp-codewhisperer/src/language-server/workspaceContext/artifactManager.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ interface FileSizeDetails {
skippedSize: number
}
const MAX_UNCOMPRESSED_SRC_SIZE_BYTES = 2 * 1024 * 1024 * 1024 // 2 GB
const MAX_FILES = 500_000

export class ArtifactManager {
private workspace: Workspace
Expand Down Expand Up @@ -175,7 +176,8 @@ export class ArtifactManager {
}

if (isDirectory(filePath)) {
const files = await glob(['**/*'], {
let fileCount = 0
const filesStream = glob.stream(['**/*'], {
cwd: filePath,
dot: false,
ignore: IGNORE_DEPENDENCY_PATTERNS,
Expand All @@ -184,7 +186,11 @@ export class ArtifactManager {
onlyFiles: true,
})

for (const relativePath of files) {
for await (const entry of filesStream) {
if (fileCount >= MAX_FILES) {
break
}
const relativePath = entry.toString()
try {
const fullPath = resolveSymlink(path.join(filePath, relativePath))
const fileMetadata = await this.createFileMetadata(
Expand All @@ -197,6 +203,7 @@ export class ArtifactManager {
} catch (error) {
this.logging.warn(`Error processing file ${relativePath}: ${error}`)
}
fileCount++
}
} else {
const workspaceUri = URI.parse(currentWorkspace.uri)
Expand Down Expand Up @@ -359,7 +366,8 @@ export class ArtifactManager {
): Promise<Map<CodewhispererLanguage, FileMetadata[]>> {
const filesByLanguage = new Map<CodewhispererLanguage, FileMetadata[]>()

const files = await glob(['**/*'], {
const files = []
const filesStream = glob.stream(['**/*'], {
cwd: directoryPath,
dot: false,
ignore: IGNORE_PATTERNS,
Expand All @@ -368,11 +376,26 @@ export class ArtifactManager {
onlyFiles: true,
})

for await (const entry of filesStream) {
if (files.length >= MAX_FILES) {
break
}
files.push(entry.toString())
}

const hasJavaFile = files.some(file => file.endsWith('.java'))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in this function when you get files, please use fast glob streaming methods. See #2003

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack let me refactor this to streaming methods

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@leigaol added the same glob.stream with early exist on MAX_FILES


for (const relativePath of files) {
const fullPath = path.join(directoryPath, relativePath)
const language = getCodeWhispererLanguageIdFromPath(fullPath)

if (!language || !SUPPORTED_WORKSPACE_CONTEXT_LANGUAGES.includes(language)) {
const isJavaProjectFile = isJavaProjectFileFromPath(fullPath)
const language = isJavaProjectFileFromPath(fullPath) ? 'java' : getCodeWhispererLanguageIdFromPath(fullPath)

if (
!language ||
!SUPPORTED_WORKSPACE_CONTEXT_LANGUAGES.includes(language) ||
// skip processing the java project file if there's no java source file
(!hasJavaFile && isJavaProjectFile)
) {
continue
}

Expand Down Expand Up @@ -631,7 +654,7 @@ export class ArtifactManager {
files: FileMetadata[]
): Promise<FileMetadata[]> {
const workspacePath = URI.parse(workspaceFolder.uri).path
const hasJavaFiles = files.some(file => file.language === 'java' && file.relativePath.endsWith('java'))
const hasJavaFiles = files.some(file => file.language === 'java' && file.relativePath.endsWith('.java'))

if (!hasJavaFiles) {
return files
Expand Down
4 changes: 0 additions & 4 deletions server/aws-lsp-codewhisperer/src/shared/languageDetection.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,6 @@ describe('LanguageDetection', () => {

describe('getCodeWhispererLanguageIdFromPath', () => {
it('should return language type with override', () => {
assert.strictEqual(getCodeWhispererLanguageIdFromPath('test/pom.xml'), 'java')
assert.strictEqual(getCodeWhispererLanguageIdFromPath('test/build.gradle.kts'), 'java')
assert.strictEqual(getCodeWhispererLanguageIdFromPath('test/build.xml'), 'java')
assert.strictEqual(getCodeWhispererLanguageIdFromPath('test/build.gradle'), 'java')
assert.strictEqual(getCodeWhispererLanguageIdFromPath('test/test.java'), 'java')
assert.strictEqual(getCodeWhispererLanguageIdFromPath('test/package.json'), 'javascript')
assert.strictEqual(getCodeWhispererLanguageIdFromPath('test/test.js'), 'javascript')
Expand Down
2 changes: 0 additions & 2 deletions server/aws-lsp-codewhisperer/src/shared/languageDetection.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -294,8 +294,6 @@ export function getCodeWhispererLanguageIdFromPath(filePath: string): Codewhispe
return 'javascript'
}

if (isJavaProjectFileFromPath(filePath)) return 'java'

for (const [extension, languageId] of Object.entries(languageByExtension)) {
if (filePath.endsWith(extension)) {
return getRuntimeLanguage(languageId)
Expand Down
Loading