Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/3307/detect-invalid-file-input #3325

Merged
merged 20 commits into from
Jun 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
41a617d
Adjust InputHelper, MergeFilter and tests for new requirements #3307
moritz-suckow May 30, 2023
ddb45cf
Add input checking to CSVExporter #3307
moritz-suckow May 30, 2023
760d5e1
Refactor InputHelper to also look for illegal folder input #3307
moritz-suckow May 30, 2023
a76194f
Refactor InputHelper, add input check to Exporter and Filters #3307
moritz-suckow May 31, 2023
06a8af4
Add input check and tests to all other parsers #3307
moritz-suckow May 31, 2023
caf0a9f
Remove empty input allowed if pipeable #3307
moritz-suckow Jun 2, 2023
a27f167
Removed stale and disabled structure modifier test #3307
moritz-suckow Jun 2, 2023
a1b894d
Fix failing MetricGardener input test #3307
moritz-suckow Jun 2, 2023
74645e4
Refactor various input file types to be File? #3307
moritz-suckow Jun 2, 2023
f92181b
Adjust null input check and TokeiImporter check #3307
moritz-suckow Jun 5, 2023
7fd724a
Add check for empty strings #3307
moritz-suckow Jun 5, 2023
7a64f36
Add empty string check for SonarImporter, remove unneeded mock #3307
moritz-suckow Jun 5, 2023
b9e8eb2
Add changelog entry #3307
moritz-suckow Jun 5, 2023
2720f6b
Adjust Sonar error message #3307
moritz-suckow Jun 5, 2023
c918279
Fix merging mistake #3307
moritz-suckow Jun 7, 2023
72802e1
Remove invalid comment and minor refactor of RepoScan check #3307
moritz-suckow Jun 7, 2023
85d34d5
Add basic input check to parser suggestions #3307
moritz-suckow Jun 7, 2023
37edd48
Fix test for Windows os
phanlezz Jun 16, 2023
c038a43
Merge main, corrected changelog
phanlezz Jun 16, 2023
b1a2410
Change error logging to exceptions to stop execution
phanlezz Jun 21, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 5 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,11 @@ and this project adheres to [Semantic Versioning](http://semver.org/)

## [unreleased] (Added 🚀 | Changed | Removed 🗑 | Fixed 🐞 | Chore 👨‍💻 👩‍💻)

### Added 🚀

- Add message outputting which parser is being configured during parser suggestions [#3335](https://github.com/MaibornWolff/codecharta/pull/3335)
- Add basic validity checking for all input resources (files/folders, url for SonarImporter) [#3325](https://github.com/MaibornWolff/codecharta/pull/3325)

## [1.118.0] - 2023-06-15

### Added 🚀
Expand All @@ -16,7 +21,6 @@ and this project adheres to [Semantic Versioning](http://semver.org/)
- Expand and restructure documentation regarding Docker usage [#3312](https://github.com/MaibornWolff/codecharta/pull/3312)
- Add current working directories as hint or default value to interactive parser and parser suggestions when asking for input [#3319](https://github.com/MaibornWolff/codecharta/pull/3319)
- Add helpful status messages when calculating parser suggestions [#3329](https://github.com/MaibornWolff/codecharta/pull/3329)
- Add message outputting which parser is being configured during parser suggestions [#3335](https://github.com/MaibornWolff/codecharta/pull/3335)

### Fixed 🐞

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import de.maibornwolff.codecharta.serialization.ProjectDeserializer
import de.maibornwolff.codecharta.tools.interactiveparser.InteractiveParser
import de.maibornwolff.codecharta.tools.interactiveparser.ParserDialogInterface
import de.maibornwolff.codecharta.tools.interactiveparser.util.InteractiveParserHelper
import de.maibornwolff.codecharta.util.InputHelper
import picocli.CommandLine
import java.io.BufferedWriter
import java.io.File
Expand Down Expand Up @@ -51,6 +52,10 @@ class CSVExporter : Callable<Void>, InteractiveParser {
throw IllegalArgumentException("depth-of-hierarchy must not be negative")
}

if (!InputHelper.isInputValid(sources, canInputContainFolders = false)) {
throw IllegalArgumentException("Input invalid file for CSVExporter, stopping execution...")
}

val projects = sources.map { ProjectDeserializer.deserializeProject(it.inputStream()) }

projects.forEach { writeUsingWriter(it, writer()) }
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
package de.maibornwolff.codecharta.exporter.csvexporter

import de.maibornwolff.codecharta.exporter.csv.CSVExporter
import de.maibornwolff.codecharta.util.InputHelper
import io.mockk.every
import io.mockk.mockkObject
import io.mockk.unmockkAll
import org.assertj.core.api.Assertions
import org.junit.jupiter.api.AfterAll
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.TestInstance
import picocli.CommandLine
import java.io.ByteArrayOutputStream
import java.io.PrintStream

@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class CSVExporterTest {
val errContent = ByteArrayOutputStream()
val originalErr = System.err

@AfterAll
fun afterTest() {
unmockkAll()
}

@Test
fun `should stop execution if input files are invalid`() {
mockkObject(InputHelper)
every {
InputHelper.isInputValid(any(), any())
} returns false

System.setErr(PrintStream(errContent))
CommandLine(CSVExporter()).execute("thisDoesNotExist.cc.json").toString()
System.setErr(originalErr)

Assertions.assertThat(errContent.toString()).contains("Input invalid file for CSVExporter, stopping execution")
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import de.maibornwolff.codecharta.serialization.ProjectSerializer
import de.maibornwolff.codecharta.tools.interactiveparser.InteractiveParser
import de.maibornwolff.codecharta.tools.interactiveparser.ParserDialogInterface
import de.maibornwolff.codecharta.tools.interactiveparser.util.InteractiveParserHelper
import de.maibornwolff.codecharta.util.InputHelper
import picocli.CommandLine
import java.io.File
import java.io.PrintStream
Expand All @@ -22,8 +23,8 @@ class EdgeFilter(
@CommandLine.Option(names = ["-h", "--help"], usageHelp = true, description = ["displays this help and exits"])
var help: Boolean = false

@CommandLine.Parameters(arity = "1..*", paramLabel = "FILE", description = ["files to filter"])
ce-bo marked this conversation as resolved.
Show resolved Hide resolved
private var source: String = ""
@CommandLine.Parameters(arity = "1", paramLabel = "FILE", description = ["files to filter"])
private var source: File? = null

@CommandLine.Option(names = ["--path-separator"], description = ["path separator (default = '/')"])
private var pathSeparator = '/'
Expand All @@ -32,7 +33,11 @@ class EdgeFilter(
private var outputFile: String? = null

override fun call(): Void? {
val srcProject = ProjectDeserializer.deserializeProject(File(source).inputStream())
if (!InputHelper.isInputValidAndNotNull(arrayOf(source), canInputContainFolders = false)) {
throw IllegalArgumentException("Input invalid file for EdgeFilter, stopping execution...")
}

val srcProject = ProjectDeserializer.deserializeProject(source!!.inputStream())

val newProject = EdgeProjectBuilder(srcProject, pathSeparator).merge()

Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,30 @@
package de.maibornwolff.codecharta.filter.edgefilter

import de.maibornwolff.codecharta.filter.edgefilter.EdgeFilter.Companion.main
import de.maibornwolff.codecharta.util.InputHelper
import io.mockk.every
import io.mockk.mockkObject
import io.mockk.unmockkAll
import org.assertj.core.api.Assertions
import org.junit.jupiter.api.AfterEach
import org.junit.jupiter.api.Assertions.assertTrue
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.TestInstance
import picocli.CommandLine
import java.io.ByteArrayOutputStream
import java.io.File
import java.io.PrintStream

@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class EdgeFilterTest {
val errContent = ByteArrayOutputStream()
val originalErr = System.err

@AfterEach
fun afterTest() {
unmockkAll()
}

@Test
fun `should create json uncompressed file`() {
main(
Expand All @@ -18,4 +37,18 @@ class EdgeFilterTest {

assertTrue(file.exists())
}

@Test
fun `should stop execution if input file is invalid`() {
mockkObject(InputHelper)
every {
InputHelper.isInputValid(any(), any())
} returns false

System.setErr(PrintStream(errContent))
CommandLine(EdgeFilter()).execute("thisDoesNotExist.cc.json").toString()
System.setErr(originalErr)

Assertions.assertThat(errContent.toString()).contains("Input invalid file for EdgeFilter, stopping execution")
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import org.junit.jupiter.api.AfterAll
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.TestInstance
import picocli.CommandLine
import java.io.File

@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class ParserDialogTest {
Expand Down Expand Up @@ -44,6 +45,6 @@ class ParserDialogTest {
val parseResult = cmdLine.parseArgs(*parserArguments.toTypedArray())
assertThat(parseResult.matchedOption("output-file").getValue<String>()).isEqualTo("sampleOutputFile")
assertThat(parseResult.matchedOption("path-separator").getValue<Char>()).isEqualTo('/')
assertThat(parseResult.matchedPositional(0).getValue<String>()).isEqualTo("sampleFile.cc.json")
assertThat(parseResult.matchedPositional(0).getValue<File>()).isEqualTo(File("sampleFile.cc.json"))
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -58,11 +58,10 @@ class MergeFilter(
)
}

val sourceFiles = InputHelper.getAndCheckAllSpecifiedInputFiles(sources)
if (sourceFiles.isEmpty()) {
logger.error("Aborting execution because one or more input files have not been found or no input was specified at all!")
return null
if (!InputHelper.isInputValid(sources, canInputContainFolders = true)) {
throw IllegalArgumentException("Input invalid files/folders for MergeFilter, stopping execution...")
}
val sourceFiles = InputHelper.getFileListFromValidatedResourceArray(sources)

val srcProjects = sourceFiles
.mapNotNull {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,20 +1,16 @@
package de.maibornwolff.codecharta.filter.mergefilter

import de.maibornwolff.codecharta.filter.mergefilter.MergeFilter.Companion.main
import de.maibornwolff.codecharta.model.Project
import de.maibornwolff.codecharta.serialization.ProjectDeserializer
import de.maibornwolff.codecharta.util.InputHelper
import io.mockk.every
import io.mockk.mockkObject
import io.mockk.unmockkAll
import io.mockk.verify
import org.assertj.core.api.Assertions.assertThat
import org.junit.jupiter.api.AfterEach
import org.junit.jupiter.api.Test
import picocli.CommandLine
import java.io.ByteArrayOutputStream
import java.io.File
import java.io.FileInputStream
import java.io.PrintStream

class MergeFilterTest {
Expand Down Expand Up @@ -98,26 +94,18 @@ class MergeFilterTest {
}

@Test
fun `should abort if at least one input file does not exist`() {
fun `should not execute merge if input is invalid`() {
mockkObject(InputHelper)
every {
InputHelper.getAndCheckAllSpecifiedInputFiles(any())
} returns mutableListOf()

mockkObject(ProjectDeserializer)
every {
ProjectDeserializer.deserializeProject(any<FileInputStream>())
} returns Project("")
InputHelper.isInputValid(any(), any())
} returns false

System.setErr(PrintStream(errContent))
CommandLine(MergeFilter()).execute(
"src/test/resources/mergeFolderTest/file1.cc.json",
"src/test/resources/mergeFolderTest/file2.cc.json",
"src/test/resources/thisDoesNotExist.cc.json").toString()
System.setErr(originalErr)

assertThat(errContent.toString()).contains("Aborting execution because one or more input files have not been found or no input was specified at all!")

verify(exactly = 0) { ProjectDeserializer.deserializeProject(any<FileInputStream>()) }
assertThat(errContent.toString()).contains("Input invalid files/folders for MergeFilter, stopping execution")
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import de.maibornwolff.codecharta.serialization.ProjectSerializer
import de.maibornwolff.codecharta.tools.interactiveparser.InteractiveParser
import de.maibornwolff.codecharta.tools.interactiveparser.ParserDialogInterface
import de.maibornwolff.codecharta.tools.interactiveparser.util.InteractiveParserHelper
import de.maibornwolff.codecharta.util.InputHelper
import mu.KotlinLogging
import picocli.CommandLine
import java.io.File
Expand Down Expand Up @@ -78,9 +79,10 @@ class StructureModifier(
private fun readProject(): Project? {
if (source == null) {
return ProjectDeserializer.deserializeProject(input)
} else if (!source!!.isFile) {
logger.error("${source!!.name} has not been found.")
return null
}

if (!InputHelper.isInputValid(arrayOf(source!!), canInputContainFolders = false)) {
throw IllegalArgumentException("Input invalid file for StructureModifier, stopping execution...")
}

val input = source!!.inputStream()
Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,29 @@
package de.maibornwolff.codecharta.filter.structuremodifier

import de.maibornwolff.codecharta.serialization.ProjectDeserializer
import de.maibornwolff.codecharta.util.InputHelper
import io.mockk.every
import io.mockk.mockkObject
import io.mockk.unmockkAll
import org.assertj.core.api.Assertions.assertThat
import org.junit.jupiter.api.Disabled
import org.junit.jupiter.api.AfterEach
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.TestInstance
import picocli.CommandLine
import java.io.ByteArrayOutputStream
import java.io.File
import java.io.PrintStream

@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class StructureModifierTest {
val errContent = ByteArrayOutputStream()
val originalErr = System.err

@AfterEach
fun afterTest() {
unmockkAll()
}

@Test
fun `reads project from file`() {
val cliResult = executeForOutput("", arrayOf("src/test/resources/sample_project.cc.json", "-r=/does/not/exist"))
Expand Down Expand Up @@ -38,16 +53,6 @@ class StructureModifierTest {
assertThat(errorStream.toString()).contains("invalid_project.cc.json is not a valid project")
}

@Disabled
@Test
fun `reads project piped input multiline`() {
val input = File("src/test/resources/sample_project.cc.json").bufferedReader().readLines()
.joinToString(separator = "\n") { it }
val cliResult = executeForOutput(input, arrayOf("-r=/does/not/exist"))

assertThat(cliResult).contains(listOf("otherFile.java"))
}

@Test
fun `returns error for malformed piped input`() {
val originalError = System.err
Expand Down Expand Up @@ -112,4 +117,18 @@ class StructureModifierTest {
assertThat(resultProject.attributeDescriptors.size).isEqualTo(3)
assertThat(resultProject.attributeDescriptors["yrloc"]).isNull()
}

@Test
fun `should stop execution if input file is invalid`() {
mockkObject(InputHelper)
every {
InputHelper.isInputValid(any(), any())
} returns false

System.setErr(PrintStream(errContent))
CommandLine(StructureModifier()).execute("thisDoesNotExist.cc.json").toString()
System.setErr(originalErr)

assertThat(errContent.toString()).contains("Input invalid file for StructureModifier, stopping execution")
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import de.maibornwolff.codecharta.serialization.ProjectSerializer
import de.maibornwolff.codecharta.tools.interactiveparser.InteractiveParser
import de.maibornwolff.codecharta.tools.interactiveparser.ParserDialogInterface
import de.maibornwolff.codecharta.tools.interactiveparser.util.InteractiveParserHelper
import de.maibornwolff.codecharta.util.InputHelper
import picocli.CommandLine
import java.io.File
import java.io.IOException
Expand Down Expand Up @@ -43,6 +44,9 @@ class CSVImporter(

@Throws(IOException::class)
override fun call(): Void? {
if (!InputHelper.isInputValid(files.toTypedArray(), canInputContainFolders = false)) {
throw IllegalArgumentException("Input invalid file for CSVImporter, stopping execution...")
}

val csvProjectBuilder = CSVProjectBuilder(pathSeparator, csvDelimiter, pathColumnName)
files.map { it.inputStream() }.forEach<InputStream> { csvProjectBuilder.parseCSVStream(it) }
Expand All @@ -54,6 +58,7 @@ class CSVImporter(
}

companion object {

@JvmStatic
fun main(args: Array<String>) {
CommandLine(CSVImporter()).execute(*args)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import de.maibornwolff.codecharta.tools.interactiveparser.InteractiveParser
import de.maibornwolff.codecharta.tools.interactiveparser.ParserDialogInterface
import de.maibornwolff.codecharta.tools.interactiveparser.util.InteractiveParserHelper
import de.maibornwolff.codecharta.translator.MetricNameTranslator
import de.maibornwolff.codecharta.util.InputHelper
import picocli.CommandLine
import java.io.File
import java.io.IOException
Expand Down Expand Up @@ -40,6 +41,10 @@ class SourceMonitorImporter(

@Throws(IOException::class)
override fun call(): Void? {
if (!InputHelper.isInputValid(files.toTypedArray(), canInputContainFolders = false)) {
throw IllegalArgumentException("Input invalid file for SourceMonitorImporter, stopping execution...")
}

val csvProjectBuilder =
CSVProjectBuilder(pathSeparator, csvDelimiter, "File Name", sourceMonitorReplacement, getAttributeDescriptors())
files.map { it.inputStream() }.forEach<InputStream> { csvProjectBuilder.parseCSVStream(it) }
Expand Down