-
Notifications
You must be signed in to change notification settings - Fork 5.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Docker-compose build writes by default "Building ..." to STDERR. #5590
Comments
You should use exit codes to detect errors. |
Yes but why is something written to STDERR when exit code = 0 ? |
We use |
Joffrey,
For me it sounds really irrational that something is dropped in error output when there is no error. |
@paulvanbladel Would being able to configure the log level of the |
Joffrey,
Not sure if the introduction of a log level would change anything. |
I have come across this as well. I genuinely think that using STDERR for warnings or separation of application and composer is a bad solution. Warning is not error and information message is not an error as well, so why are we using STDERR? |
…se to overcome compose issue See more about this issue here: docker/compose#5590.
…by integration tests (#9) * Prepare the grounds for using Docker Compose when running integration tests * Improve script used for checking container health state * Add support for enabling printing info for debugging purposes * Integration tests are now targeting a database run via Docker Compose * Fix incorrect YAML property * Remove comments from YAML fragment as Azure DevOps complains about them * Fix path pointing to Docker Compose file * Add pipe symbol as requested by Docker Compose task * Enable debugging for Docker Compose services * Use hard-coded sleep to ensure compose services are up when running IT * Fix YAML indentation * Give enough time to PostgreSQL DB to start before running IT * Tweak Docker Compose task parameters * Ensure compose services are healthy before running IT * Fix Docker Compose command used for checking service status * Use PowerShell Core to interact with Docker Compose * Fix passing environment variables to PowerShell Core script * Fix PowerShell Core script syntax * Fix Azure DevOps logging command syntax used for displaying info messages * Temporarily hard-code host port for the db compose service * Provide a default database port for the template job file * Log database host port for debugging purposes * Identify the Docker host port associated with the 'db' compose service * Publish test log file for debugging purposes * Publish test log file for debugging purposes * Temporarily log IT connection string for debugging purposes * Fix typo in parameter reference * Remove logging connection string as CI pipeline has been fixed * Fix typo * Use PowerShell script to start compose services used by IT * Remove no longer needed build step * Wrap environment variable values with single quotes * Remove > symbol since PowerShell was misinterpreting it * Split PowerShell long line via tick symbol * Un-ingore .env file to be used by Docker Compose * Avoid breaking build due to Docker Compose writing to standard error * Fix referencing variables storing connection strings to be used when running IT * Fix variable name by adding missing dot symbol * Fix referencing variables in steps related to running IT * Move PowerShell script used for starting compose service to a better location * Move PowerShell script used for starting compose service to a better location * Move PowerShell script used for starting compose service to a better location * Move PowerShell script used for starting compose service to a better location * Remove invalid string * Add comments and validation rules to PowerShell script parameters * Fix displaying message about mapping container port to host one * Fix displaying variable value after mapping container port to host one * Made Docker Compose service names more explicit * Add macOS agent * Fail script if Docker Compose writes anything to standard error stream * Fail build in case compose services fail to start * Fix handling Docker Compose command standard streams * Fix redirecting all streams to null * Fix redirecting success,warning and error streams to null * Temporarily disable build on macOS agents till the pipeline is working as expected * Replace redirecting Docker Compose streams with --log-level flag * Replace redirecting Docker Compose streams with --log-level flag, take 2 * Replace redirecting Docker Compose streams with --log-level flag, take 3 * Replace redirecting Docker Compose streams with --log-level flag, take 4 * Detect errors while running Docker Compose via $LASTEXITCODE * Fix exit condition when everything it's OK * Re-enable macOS agent * Install Docker Compose via brew on macOS agent * Revert commit since compose issue occurring on macOS agent is caused by older Docker version * Use compose file v3.7 since macOS agent cannot run a newer version of Docker compatible with v3.8+ * Fix incrementing the counter used when checking whether Docker service has started * Enable Windows-based agent too * Fix validating input parameter * Parameterize the name of the Docker image used for running the DB to be targeted by IT * Fix passing arguments to PowerShell script * Explicitly set string properties to avoid issues due to invalid types * Fix reference to compose variable * Log more info for debugging purposes * Log more info for debugging purposes, take 2 * Wrap parameter values with single quotes * Explicitly set job parameters to avoid pipeline considering them not present * Remove no longer needed logging messages * Fix parameter name in XML doc * Disable ReSharper warning about unused variable since it will be used at some point * Add new word to project dictionary * Avoid possible NullReferenceException when running integration tests * Disable ReSharper warning since I'm testing whether the constructor will throw an exception * Since compose services are run as part of CI pipeline, no need for attaching a console * Add extra link to article describing the meaning of 'xmin' property in an EF Core entity * Remove redundant environment variables * Cache NuGet packages when requested only * Remove unused 'databasePort' job parameter * Remove extra whitespaces * Restoring NuGet packages happens only in case there is no cache for them * Fix condition by adding missing round bracket * Fix condition used for evaluating whether restoring NuGet packages is performed or not * Remove whitespaces since they break the commands * Remove job parameters default values since they bring no real value right now * Update .NET Core SDK to v3.1.302 * Update ReportGenerator to v4.6.4 * Rollback updating ReportGenerator to v4.6.4 since Azure DevOps feed doesn't have it yet * Try installing latest available version of Docker via homebrew * Rollback installing latest version of Docker since macOS based agent does not support it * Replace hard-coded value with variable * Set default value for job parameter to avoid code duplication * Log connection string pointing to test database for debugging purposes * The databaseHost job parameter must be explicitly set * Experiment with specifying volume for the DB to be targeted by integration tests * Experiment with specifying volume for the DB to be targeted by integration tests, take 2 See more about this approach here: https://stackoverflow.com/a/55366707 * Experiment with specifying volume for the DB to be targeted by integration tests, take 3 * Fix Docker Compose volume related error occurring on Windows based agents See more here: docker/compose#5842 (comment). * Add missing key-value pair separator * Wrap environment variable in single quotes hoping things will work out :) * Experiment with specifying volume for the DB to be targeted by integration tests, take 4 * Set volume path according to the Docker container (Linux or Windows) * Make script starting compose services more robust * Fix splitting long line in PowerShell script * Make script starting compose services more robust, take 2 * Fix creating a custom object in PowerShell * Fix Docker Compose command broken due to formatting code * Fix referencing true and false boolean values * Fix referencing container id when fetching port mappings * Wrap container port with double quotes hoping port mappings will be fetched on Windows * Specify host port to avoid Windows-based Azure DevOps agents not fetching any port mappings * Let again Docker assign a dynamic host port * Replace identifying ports via 'docker port' with 'docker inspect' * Fix variable name * Fix fetching all JSON properties * Create Docker volume before starting compose services AFAIK, creating the volume may be needed before starting compose services on Windows-based Azure DevOps agents since PostgreSQL complains that its folder has different permissions than the expected ones * Fix declaring named volume * Explicitly set durations used by Docker Compose * Wrap strings in double quotes * Rename job parameter to increase code readability * Display message about creating Docker volume to make the build step more transparent * Fix passing job parameter as PowerShell script parameter value * Ignore comments and empty lines found in Docker Compose environment file * Expose Docker Compose logs as build artifact for debugging purposes * Let the PowerShell script handle errors * Publish Docker Compose logs as build artifact * Publish Docker Compose logs as build artifact, take 2 * Display Docker Compose logs * Publish Docker Compose logs via build step * Fix splitting long line into multiple shorter ones * Fix publishing Docker Compose logs * Create missing folders if needed when writing Docker Compose log file to disk * Create missing folders if needed when writing Docker Compose log file to disk, take 2 * Ensure Docker Compose writing to standard error stream does not break script * Remove no longer needed Azure DevOps command * Add compose project name to the name of the compose service log file being published * Add missing parameter when creating Docker volumes * Print Azure DevOps-style errors when Docker commands fail * Add timestamps to the published Docker Compose logs * Avoid checking whether commands have ended with success since they write to stderr * Fix container path passed as job parameter value * Publish compose service log files in a folder per Azure DevOps agent OS * Fix container path * Fixcommand used for creating compose service volume * Make use of PGDATA env var in order to fix Windows-based agent error More specifically, this is the output of the db4it compose service: 2020-08-08T18:00:53.256999700Z initdb: error: could not access directory "C:/pgsql/data": Permission denied 2020-08-08T18:00:53.292312700Z Access is denied. 2020-08-08T18:00:53.292312700Z Access is denied. 2020-08-08T18:00:53.292312700Z The system cannot find the drive specified. 2020-08-08T18:00:55.477969200Z psql: error: could not connect to server: could not connect to server: Connection refused (0x0000274D/10061) 2020-08-08T18:00:55.477969200Z Is the server running on host "localhost" (::1) and accepting 2020-08-08T18:00:55.477969200Z TCP/IP connections on port 5432? 2020-08-08T18:00:55.477969200Z could not connect to server: Connection refused (0x0000274D/10061) 2020-08-08T18:00:55.477969200Z Is the server running on host "localhost" (127.0.0.1) and accepting 2020-08-08T18:00:55.477969200Z TCP/IP connections on port 5432? * Fix container path * Fix path pointing to test related artifacts to be published * Fix container path, take 2 * MOve .artifactignore file to repo root, as recommended by official doc See more here: https://docs.microsoft.com/en-us/azure/devops/artifacts/reference/artifactignore?view=azure-devops * Fix container path, take 3 Fix error: ERROR: for db4it Cannot create container for service db4it: invalid volume specification: 'db4it_data:C:\\db4it:rw' * Simplify folder path storing published compose service log file * Tune a little bit the health check durations * Print message for debugging purposes * Rephrase message to include compose project name * Tune a little bit printing message for debugging purposes * Fix printing message * Fix printing another message * Ensure no PowerShell errors goe unnoticed * Need to set errorActionPreference to Continue and failOnStderr to False to overcome compose issue See more about this issue here: docker/compose#5590. * Remove db4it volume - let's see what happens * Remove any trace of Docker volume since PostgreSQL compose service does not start on Windows * Polish messages printed for debugging purposes * Refine publish artifact exclusion and inclusion rules * Reduce number of if-statements * Move .artifactignore file under Tests folder to limit what gets published * Change build step id and name to better reflect its purpose * Ensure compose project name is unique across each builds * Use minimum amount of info to ensure unique compose project names across all builds * Remove no longer used variable * Rename compose project name to follow the Azure DevOps user-defined variable naming convention See this convention here: https://docs.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#variable-characters. * Make compose project name even shorter * Ensure compose project name contains letters only, as using numbers (e.g. build ID) does not work * Remove job parameter duplication * Ensure job parameter is set outside job template file to avoid considering it empty string * Revert compose project name to last working value * Specify job parameter value outside job template to ensure it will not be seen as empty string * Fix variable name used as example * Change variable name to correctly state its purpose * Temporarily expose more ports for testing purposes * Revert multiple ports to just one * Make use of 'docker container <VERB>' commands * Declare external volume for the compose service * Remove volume since build fails on Windows
Please fix this. Its making all my pipelines to fail |
Docker-compose build writes by default "Building MyImage" to STDERR where MyImage is the image I'm building.
That makes integration with a build process quite difficult, since I want to capture real errors.
Any workaround possible ?
The text was updated successfully, but these errors were encountered: