Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for YAML files to @PropertySource [SPR-13912] #18486

Closed
spring-projects-issues opened this issue Feb 3, 2016 · 24 comments
Closed

Add support for YAML files to @PropertySource [SPR-13912] #18486

spring-projects-issues opened this issue Feb 3, 2016 · 24 comments

Comments

@spring-projects-issues
Copy link
Collaborator

@spring-projects-issues spring-projects-issues commented Feb 3, 2016

Rob Moore opened SPR-13912 and commented

As the Spring documentation notes, the @PropertySource annotation does not support YAML files so one must use properties files in these cases.

This can be especially annoying with third party libraries using these annotations as a separate properties file must be created.


Affects: 4.2.4

Reference URL: https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html#boot-features-external-config-yaml-shortcomings

18 votes, 27 watchers

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented May 13, 2016

Stéphane Nicoll commented

Another use case is using i18n with yml (i.e. messages.yml)

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Jun 15, 2016

Keith Hoopes commented

Here's our use case:

  1. We are hosting multiple war files on a single tomcat instance (not using boot runner).
    a. This means we cannot use the spring.config.location and spring.config.name JVM parameters or environment variables.
  2. Configuration files are not allowed to be included with the war
  3. Any configuration that needs to be modified depending on the environment or server (dev, test, prod for example), must be contained in an external configuration file.
  4. Profiles are not allowed (we've had problems with certain admins running apps in prod mode in test, and test mode in prod).
  5. Ops/IT must be allowed to specify the location of all configuration files using environment variables that are only accessed by (or are unique to) the particular application to which it belongs.

Example:

// snip: package and import statements
@SpringBootApplication
@PropertySource("file:${myapplication.cofnig.url}"))
public class MyApplication expends SpringBootServletInitializer{
    //snip: content, main, init methods etc...
}

Right now, this approach only works for properties files. It would be really convenient if this would work for either, and let IT decide on the format they best prefer. The annotation should be format agnostic.

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Dec 9, 2016

Gregor Frey commented

I support this request. I used the locations attribute in the ConfigurationProperties annotation, which allowed property files in yaml format. As this attribute is deprecated since SpringBoot 1.4 I'm looking for a simple alternative. To allow yaml files in the PropertySource annotation would be quite helpful.

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Dec 12, 2016

Stéphane Nicoll commented

Gregor Frey there is a reason why it's deprecated, perhaps you could take that into account in your own arrangement? I think there are better options in Spring Boot than this (which means you wouldn't have to wait for this particular feature). An EnvironmnentPostProcessor is a far more powerful option if you don't want to use boot's ways of configuring the environmnent.

But I digress, this is a useful feature in the core indeed.

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Dec 12, 2016

Gregor Frey commented

Hi Stéphane, thanks for pointing me to the EnvironmentPostProcessor. I will definitely have a look.

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Feb 24, 2017

Emerson Farrugia commented

Is there any momentum on this? It seems bizarre that Spring Boot doesn't have a way to compose configuration properties using YAML, which is a first-class configuration mechanism. The justification seems to be that the Spring Framework itself doesn't support it.

My use case is quite simple. I have a library I share amongst microservices to configure logging. I want that same library to contain YAML configuration for logging, e.g. setting logger levels or configuring trace endpoints. Similar examples could be data layer libraries that contain database configurations.

To use YAML, I'm currently limited to using Spring Boot's spring.config.location. Using @PropertySource seems like a saner option.

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Feb 24, 2017

Stéphane Nicoll commented

Ironically enough, that will never work.

The logging system is initialized very early on (before the ApplicationContext is even created). So any @PropertySource you put in your code will be processed way after the logging system has fully initialized. That's the reason there isn't a high incentive to implement this: in a Spring Boot context in particular, @PropertySource is very limited and Spring Boot offers much better alternatives, including EnvironmentPostProcessor that would work with the use case you're describing.

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented May 10, 2017

Roman Rodov commented

Everyone keeps speaking of these "better alternatives" in Spring Boot :) What exactly are they?

spring.config.location and spring.config.name can be used in the top level application, sure. What about library modules setting these default properties in a yml file and then letting the top level application override them (if needed, or adapt the value from the base module)? How can it be achieved using spring boot without having to do anything in the top level app?

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Jun 21, 2017

Michael Johnson commented

Just a thought. Is there any chance this could be done via a maven plugin that would convert yaml files to properties files at build time?

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Jun 27, 2017

Stéphane Nicoll commented

After much consideration, we're inclined to reject this request.

It feels to us that most voters are using Spring Boot and using @PropertySource is definitely something that we do not recommend. I've just expanded the documentation to include a specific example of loading a YAML file in the environment that should answer the concern.

I am not closing this issue right the way. If you are using Spring Boot and that doesn't answer your concern, please join us on Gitter and we can chat about it. If you're not using Spring Boot, please expand as why you need this at the @PropertySource level.

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Jul 3, 2017

Keith Hoopes commented

I see why @PropertySource would not be proper for loading spring boot auto configuration properties. So really, shouldn't we deprecate the annotation all together
and perhaps add a different one that would allow for the property to be loaded before the application context? The example is definitely workable, but it feels somehow out of place in a spring boot application that should "just know" that an "application.yml" file needs to be loaded before the application context.

My problem is that I have to deploy my applications as war files on Tomcat in windows (not my decision), so I have multiple springBoot applications that share the same JVM parameters. This means that I cannot use "-Dspring.config.location=somedir/whatever.properties" nor "-Dspring.config.name=myapp.application.yml" to indicate application properties.

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Jul 4, 2017

Stéphane Nicoll commented

Let's not discuss Spring Boot use case here as I've indicated that we don't consider @PropertySource to be a good fit in light of what Spring Boot has to offer. Deprecating the annotation is not an option (there is really no reason) and replacing it with another annotation is not going to fix anything (same limitation of processing the user configuration once certain things like logging have already completed).

As for your use cases, we can discuss them in the Gitter Spring Boot channel but it boils down to the fact that if you can hardcode the location in your code, you can also hardcode whatever option Spring Boot offers using SpringApplicationBuilder. While @PropertySource may sound interesting because "you add an annotation and you're done", it really doesn't do that so I don't want to increase that impression by offering yaml support for it.

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Jul 4, 2017

Dominik Bartholdi commented

@snicoll do you care to explain why YAML is not going to be supported by @PropertySource? I've read your comments, but I don't understand the rational behind the decision, just because its not to be used in a Boot App does not make the request useless. After all its just a different format and many people like YAML a lot more then the normal Properties format - supporting it does not hurt and its easy to add.

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Jul 6, 2017

Keith Hoopes commented

@snicoll Even though it would not help me do what I was intending, I still agree with @domi. I think perhaps what I was trying to do confused the issue, since I was not understanding the intended use of 'PropertySource'. Since there's no reason to deprecate it, then there's no reason to not enhance it either. Just because some silly developer doesn't understand the best or intended uses, doesn't mean it shouldn't be implemented for those that do. It's the principle of coding with the assumption that everyone knows what they are doing.

I am willing to work on it. Would you accept, or consider accepting, a pull request to add this feature?

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Jul 6, 2017

Stéphane Nicoll commented

Since there's no reason to deprecate it, then there's no reason to not enhance it

I disagree. The whole discussion here is about Spring Boot and how we can "easily" add custom YAML files to the environment. Easy doesn't necessarily means correct. In the context of Spring Boot, the whole use of @PropertySource (be it for properties or potentially yaml based format) is kind of inconsistent and we don't want to promote that feature.

So @PropertySource is a framework feature and for the applications out there using Spring Framework, that's a valid solution. Just because we've rightfully decided to support those users doesn't mean that we have to improve it for new use cases.

Sorry, but we won't accept a PR since we've decided to reject this request.

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Jul 6, 2017

Keith Hoopes commented

Well, thanks for talking with me about it at least. It's been engaging and enjoyable. I'll just leave this last "little" thought and be done.

I was focused only on the Spring Framework here; I had thought we moved away from discussing SpringBoot since you asked to move that conversation to gitter.

In the context of Spring Framework only, it makes sense to include YAML (and it's superset JSON?) in order to provide a complete, quality api.
It does not make sense (to me) to allow foreign/outside "wrapper" frameworks to influence the quality and usefulness of the base framework being developed.

In the context of SpringBoot, I don't see why this would cause a problem or send the wrong message; I disagree with the philosophy of preventing developers from doing stuff I don't like (even if I am right).

I won't touch on the discussion of how to specifically load individual SpringBoot configurations for each war in Tomcat, since you and others have provided some information for me I neeed to think about and upon which I need to experiment.

You should probably update the documentation for SpringBoot.

https://docs.spring.io/spring-boot/docs/current/reference/html/howto-properties-and-configuration.html#howto-change-the-location-of-external-properties

I read it several times to be sure, and I had others read it without any prior input, and everyone thought that "If YAML is used then files with the ‘.yml’ extension are also added to the list by default." means that '@PropertySource' already supports YAML.
It also states that properties "are added to the Environment early enough to be used in all phases of the ApplicationContext lifecycle", but I am not able to externalize the properties if I need to use all aspects for Spring Boot configuration. Since that is "Spring Boot" documentation, developers would expect it to work for SpringBoot configuration, so it needs some clarification and an update to let us know it is not recommended nor can it be used to load SpringBoot configuration.without any clarifications.

There is documentation at "https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#boot-features-external-config-yaml-shortcomings" that add some clarity, so perhaps a link could be added.

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Jul 6, 2017

Keith Hoopes commented

I would also be willing to work on a PR to update the documentation according to your input/direction if you would like.

https://github.com/Pytry

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Dec 19, 2017

Christian Oestreich commented

I know I am late to this party but our use case is a bit of a case-study in why the support for multiple yml files is important to us.

We make extensive use of Spring Rest Docs and we import a lot of code snippets, test documentation and configuration into the generated site docs. The readability of YML is vastly superior to that of properties files due to its use of spacing to help delineate hierarchy. We have attempted to move all our spring cloud stream bindings to a "bindings.yml" file and include this file into the application via a @PropertySource and in asciidocs via a source include::/bindings.yml. This clearly failed for the reasons above and we had to convert to using a @PropertySource with a bindings.properties file.

Contrary to your above points, we very poignantly want to commonly split configuration up across multiple files so we can use these configuration snippets directly into our documentation and always have accurate and up to date docs. This requires that we externalize and make extensive use of the @PropertySource to pull these in.

I feel that your opinions on the validity of this type of implementation are short sighted and don't take into account creative use scenarios and the fact that YML is a better documentation format and visually easier to read.

Which one would you want to write, maintain, is easier to read and works better with IDEs?

spring:
  cloud:
    stream:
      bindings:
        cedm-consumer:
          destination: xyz-ps-cedm-exchange
          group: ps
          contentType: text/plain
          consumer:
            concurrency: 3
        cedm-producer:
          destination: xyz-ps-cedm-exchange
          group: ps
          contentType: text/plain
        core-input:
          destination: xyz-ps-input-exchange
          group: ps
          contentType: text/plain
          producer:
           requiredGroups: ps
        core-output:
          destination: xyz-ps-output-exchange
          group: ps
          contentType: text/plain
          consumer:
            concurrency: 20
        response:
          destination: xyz-ps-response-status-exchange
          group: ps
          contentType: text/plain
          producer:
            requiredGroups: ps
        error-queue-app:
          destination: xyz-error-queue
          group: ps
          contentType: text/plain
        core-error:
          destination: core-ps-error
          group: ps
          contentType: text/plain
          consumer:
            concurrency: 5
        xyz-input-error:
          destination: xyz-ps-error
          group: ps
          contentType: text/plain
          consumer:
            concurrency: 5
        xyz-output-error:
          destination: xyz-ps-error
          group: ps
          contentType: text/plain
spring.cloud.stream.bindings.cedm-consumer.destination=xyz-ps-cedm-exchange
spring.cloud.stream.bindings.cedm-consumer.group=ps
spring.cloud.stream.bindings.cedm-consumer.contentType=text/plain
spring.cloud.stream.bindings.cedm-consumer.consumer.concurrency=3
spring.cloud.stream.bindings.cedm-producer.destination=xyz-ps-cedm-exchange
spring.cloud.stream.bindings.cedm-producer.group=ps
spring.cloud.stream.bindings.cedm-producer.contentType=text/plain
spring.cloud.stream.bindings.core-input.destination=xyz-ps-input-exchange
spring.cloud.stream.bindings.core-input.group=ps
spring.cloud.stream.bindings.core-input.contentType=text/plain
spring.cloud.stream.bindings.core-input.producer.requiredGroups=ps
spring.cloud.stream.bindings.core-output.destination=xyz-ps-output-exchange
spring.cloud.stream.bindings.core-output.group=ps
spring.cloud.stream.bindings.core-output.contentType=text/plain
spring.cloud.stream.bindings.core-output.consumer.concurrency=20
spring.cloud.stream.bindings.response.destination=xyz-ps-response-status-exchange
spring.cloud.stream.bindings.response.group=ps
spring.cloud.stream.bindings.response.contentType=text/plain
spring.cloud.stream.bindings.response.producer.requiredGroups=ps
spring.cloud.stream.bindings.error-queue-app.destination=xyz-error-queue
spring.cloud.stream.bindings.error-queue-app.group=ps
spring.cloud.stream.bindings.error-queue-app.contentType=text/plain
spring.cloud.stream.bindings.core-error.destination=core-ps-error
spring.cloud.stream.bindings.core-error.group=ps
spring.cloud.stream.bindings.core-error.contentType=text/plain
spring.cloud.stream.bindings.core-error.consumer.concurrency=5
spring.cloud.stream.bindings.xyz-input-error.destination=xyz-ps-error
spring.cloud.stream.bindings.xyz-input-error.group=ps
spring.cloud.stream.bindings.xyz-input-error.contentType=text/plain
spring.cloud.stream.bindings.xyz-input-error.consumer.concurrency=5
spring.cloud.stream.bindings.xyz-output-error.destination=xyz-ps-error
spring.cloud.stream.bindings.xyz-output-error.group=ps
spring.cloud.stream.bindings.xyz-output-error.contentType=text/plain

Every developer I have asked on our team prefers to look at and use the YML, especially in documentation. Have you opened this discussion up to the broader audience and allowed community input via a poll or other mechanism concerning supporting YML files and @PropertySource?

So @PropertySource is a framework feature and for the applications out there using Spring Framework, that's a valid solution. Just because we've rightfully decided to support those users doesn't mean that we have to improve it for new use cases.

This comment just makes no sense and is like saying: "Well... we invented a wheel, but you can only use it for wheelbarrows as we don't feel that it's usefulness extends beyond that."

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Jun 26, 2018

Claudiu Alexandru Popa commented

Sorry. I know it's a little bit to late... Or, on the contrary, it's still very valid...

But, we are now (at the time I am writing this) at Spring version 5.0.7, and @PropertySource still exists in spring.

All the discussion is not, in my opinion, about spring-boot. It is about Spring Framework

And since we have @PropertySource in Spring Framework, which can be used with a .properties file, I see no reason to not be used with an .yaml (.yml) file.

That's why, until the @PropertySource is not completely removed, I consider that the feature of using it with properties file or yaml file (at user disposal's) should be offered.

I have already implemented (just for myself) this.

Please, can you evaluate the possibility to integrate my changes into the main branch?

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Jan 12, 2019

Bulk closing outdated, unresolved issues. Please, reopen if still relevant.

@mistriel
Copy link

@mistriel mistriel commented Oct 29, 2019

I strongly agree with the supporters of this feature !
YAML files should be supported as 1st class citizens in Spring F.W.

Should be reopened.

@maxmin13

This comment was marked as disruptive content.

@mazine-rr
Copy link

@mazine-rr mazine-rr commented May 19, 2020

.yaml is far more enjoyable to write, read and modify than .properties. Please reopen.

@raahool

This comment was marked as off-topic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants