[INFRA-2751] Team Foundation Server Plugin uses closed-source dependency

[jenkins-infra-bot]

It appears that there is no publicly available source code for https://github.com/jenkinsci/tfs-plugin/tree/master/tfs-sdk/src which means the plugin is not OSI-approved OSS "all the way down" as required by https://www.jenkins.io/project/governance/#license

TFS plugin distribution needs to be suspended and the plugin removed from the setup wizard at https://github.com/jenkinsci/jenkins/blob/eb0876cdb981a83c9f4c6f07d1eede585614612a/core/src/main/resources/jenkins/install/platform-plugins.json#L72

---

Originally reported by danielbeck, imported from: Team Foundation Server Plugin uses closed-source dependency

• status: Closed
• priority: Minor
• resolution: Fixed
• resolved: 2020-11-14T16:01:06+01:00
• imported: 2022/01/10

[jenkins-infra-bot]

danielbeck:

This might be fun, I have no idea how the setup wizard handles a plugin that isn't distributed anymore.

[jenkins-infra-bot]

oleg_nenashev:

No idea either. Apart from that, the potential impact on active users is quite high (6000 installations). Before depublishing, I would recommend contacting the plugin maintainers and checking with them whether they could fix it in an expedited way. If no response in 48 hours, +1 for depublishing.

CC Alex Earl

[jenkins-infra-bot]

danielbeck:

I wasn't able to find the source code, and https://www.microsoft.com/en-us/download/details.aspx?id=47727 seems to be the official download.

The redist/lib/com.microsoft.tfs.sdk-14.0.3.jar is the same file as in the plugin repo.

The license is clearly non-open.

=================================
TFS SDK for Java Redistributables
=================================
You may redistribute the following files under the

Team Foundation Server Software Development Kit for Java

Software License Terms.
ThirdPartyNotices.html
lib/com.microsoft.tfs.sdk-14.0.3.jar

…

and

You may not

• work around any technical limitations in the software;
• reverse engineer, decompile or disassemble the software, except and only to the extent that applicable law expressly permits, despite this limitation;
• make more copies of the software than specified in this agreement or allowed by applicable law, despite this limitation;
• publish the software for others to copy;
• rent, lease or lend the software;
• transfer the software or this agreement to any third party; or
• use the software for commercial software hosting services.

[jenkins-infra-bot]

danielbeck:

Olivier Dagenais Kellie Jos leah antkiewicz Jason Prickett Sriram B As maintainers, please review this issue and the previous comments. I am currently planning to suspend distribution of TFS Plugin in a few days otherwise.

[jenkins-infra-bot]

danielbeck:

jenkins-infra/update-center2#455

[jenkins-infra-bot]

leantk:

Hi all,

Thanks for letting us know the issue. To clarify, the source code (not just the jar) needs to be opened sourced to keep the plugin from being removed? If it is removed, what happens to the users that have the plugin installed already? I'm looking at this with my team now to see how easy it is for us to make this happen and I can get back to you.

 

Thanks,

-Leah

[jenkins-infra-bot]

slide_o_mix:

Isn't there a method that the plugin could download that dependency (or have users download the dependency) and place it in the classpath? I seem to remember that is what other plugins have done, but I am not sure it would work in this case.

[jenkins-infra-bot]

jthompson:

The liquibase-runner plugin recently went through something like this. I didn't follow all the details and the resolution, but there was discussion on the dev email list and INFRA-2622.

[jenkins-infra-bot]

danielbeck:

To clarify, the source code (not just the jar) needs to be opened sourced to keep the plugin from being removed?

The rule is that we're not distributing closed-source bits via update sites. Plugins can download them however (perhaps once users agree to licensing terms). Two approaches I've seen:

• Download a library from a (potentially configurable) URL as a precondition for the plugin to work. I think https://plugins.jenkins.io/liquibase-runner/ recently switched to this approach (using ToolInstallation, which may or may not make sense)
• Have an "installer" plugin that sets up a third party update site in Jenkins, from which the "real" plugin can be downloaded. https://github.com/jenkinsci/cloudbees-plugin-gateway did that, many years ago.

In terms of "how to get there from here", the first option is probably easier to do.

If it is removed, what happens to the users that have the plugin installed already?

It continues working. We're not removing software from users' systems.

They'll need to jump through hoops to reinstall the plugin though, which might affect some environments built on ephemeral Jenkins instances.

[jenkins-infra-bot]

danielbeck:

leah antkiewicz Any news here?

[jenkins-infra-bot]

danielbeck:

jenkinsci/jenkins#5016 to remove the plugin from the setup wizard.

[jenkins-infra-bot]

ianw:

As a user of TFS for almost 8 years, I am not entirely surprised Microsoft and leah antkiewicz have not responded to your requests. It seems since TFS 2015 and the introduction of robust support for Git, MS have frozen development of the TVFC side of TFS. Since the acquisition of GitHub, they appear to have abandoned support as well. It would be nice if they would actually at least respond with some useful information.

Meantime, here's my understanding...
jenkinsci / tfs-plugin source incorporates the aforementioned com.microsoft.tfs.sdk-14.0.3.jar .
tfs.sdk is available for public download as part of Microsoft Team Explorer Everywhere 2015.

I believe the tfs.sdk source comes from microsoft / team-explorer-everywhere, specifically: build/modules/sdk (source)

I am no expert in copyright, but the team-explorer-everywhere/LICENSE.txt would seem to suggest it is "Open Source"; not sure if that means OSI-compliant.

In early 2016, MS released this announcement relating to Team Explorer Everywhere:

Microsoft Open-sources the Team Explorer Everywhere Eclipse Plugin for Visual Studio Team Services and Team Foundation Server

Microsoft is making available its plugin for Eclipse, Team Explorer Everywhere (TEE), for both Visual Studio Team Services and Team Foundation Server (TFS) as open source software on GitHub. The announcement was made today by Shanku Niyogi, General Manager Developer Division, during the keynote at EclipseCon in Reston, Virginia, when he also announced Microsoft is joining the Eclipse Foundation as a Solutions Member. By Microsoft joining the Foundation, the Team Explorer Everywhere plugin can now be easily discovered and installed from within the IDE in the Eclipse Marketplace under the Help menu.

The Team Explorer Everywhere source code joins the Azure Toolkit for Eclipse as open source software from Microsoft for Eclipse. By joining the Eclipse Foundation and open-sourcing Team Explorer Everywhere, Microsoft continues its commitment to support any developer, any language and any platform.

Developers can learn more about the Team Explorer Everywhere Eclipse plugin on the Java-focused pages at java.visualstudio.com. More about Azure support for Java developers can be found at azure.com/java.

I don't know how you handle a scenario where the source for the packaged jar has apparently been released to the community subsequent to its release and inclusion. Does that apply retroactively?

As I mentioned, I do get the feeling MS have abandoned support for the TFVC aspect of TFS and left it "to the community".

Personally, I really would have liked MS to actually deal with some longstanding and critical issues, including SECURITY-1506 / CVE-2020-2249: tfs Plugin 5.157.1 and earlier stores a webhook secret unencrypted in its global configuration file,. Their last significant release for tfs-plugin was 5.133.0, Mar 2018.

ps: Also opened WEBSITE-754 to document the handling of removed plugins ("forbidden") is confusing to the users who know the plugin used to exist at plugins.jenkins.io and is now not resolving.

[jenkins-infra-bot]

ianw:

The following does not really support the suggestion that IF tfs-plugin IS license compliant, it should be reinstated in Jenkins plugins repo.

MS have also evidently abandoned support for Azure Repos Extension for Visual Studio Code. That is the the support for TFVC within VS Code. The DEPRECATED.md notice appears Nov 6, 2020, when they "unpublished the extension from the Visual Studio Code Marketplace."

MS may have simply forgotten (or can't be bothered) to get around to requesting it be dropped from the Jenkins plugin repo.

Nevertheless, if not a license issue, is should remain and let them deal with the rest.

[jenkins-infra-bot]

danielbeck:

The plugin's distribution has been suspended a week ago. This issue is resolved.

[jenkins-infra-bot]

ianw:

With respect, if the distribution was suspended because the license conditions could not be ascertained and MS was non-responsive, I believe you now have the information as to the source of origin and license. That should merit reevaluation from a compliance and availability perspective. Should an issue be filed to re-evaluate and reinstate?

[jenkins-infra-bot]

danielbeck:

I believe you now have the information as to the source of origin and license. That should merit reevaluation from a compliance and availability perspective.

If it's as simple as "here's the MIT licensed source code" I expect that the maintainers would have told us at any time in the last 5 weeks. That they didn't indicates it's not that easy, or there's no interest in having the plugin published, which is a problem in itself. I see no reason to restore distribution of the plugin at the current time, with unresponsive maintainers and a published unresolved security vulnerability.

Re WEBSITE-754, that's the wrong issue link…? But to respond to the issue as you describe it: We generally add a deprecation notice that includes an explanation for why a plugin was suspended. This feature exists since Jenkins 2.248. I haven't done this in this case because I wasn't confident (still am not) that this is the final state, and then calling it deprecated is premature. I'll probably do that in a few weeks unless we hear back from maintainers.

[jenkins-infra-bot]

ianw:

 My bad on WEBSITE-754, should be: WEBSITE-764. That issue had nothing to do w/TFS, rather the loss of PMD.

As for the TFS plugin, I suspect it's "there's no interest in having the plugin published, which is a problem in itself". Unfortunately, that leaves no ready options to connect Jenkins to TFS. I hope MS do eventually provide a clarifying response. At least the source of packaged jar is documented in this thread.

[jenkins-infra-bot]

timja:

Hi leah antkiewicz any news on this?

[jenkins-infra-bot]

mtughan:

Daniel Beck, if the TFS plugin will not be reinstated due to the issues above, that'll create a problem for LTS users who have the TFS plugin if JENKINS-64241 is correct that the TFS plugin has a tables-to-divs regression in it. If the source code is determined to be publicly-available and MIT-licensed at microsoft/team-explorer-everywhere, what else would be required to have the plugin distributed again? An active maintainer and resolution of the open security vulnerability?

[jenkins-infra-bot]

JIRAUSER131715:

Daniel Beck – this plug in is needed for my team to function as well. 

 

Michael Tughan makes an excellent point and has very valid questions. Can we reconsider fixing this plugin. The refusal of JenkinsSCI to update this means we are going to have to stay on older, insecure, versions of Jenkins because of a political pissing match. 

[jenkins-infra-bot]

oleg_nenashev:

Hello, Jenkins Governance Board member and a former TFS user here. Please let me clarify a few bits.

Current state

At the moment the recommended workaround for users is to build the plugin on their own OR to setup a custom update site which would serve it within their infrastructure (e.g. using Juseppe: https://github.com/jenkinsci/juseppe or another update center implementation). The plugin and its codebase remains open-source, and the plugin binaries are still available in the Jenkins Artifactory instance. Anyone can deploy this plugin on their instance if they want to do so.

What we do as the Jenkins project is that we refuse to host it within our update centers. Closed-source dependency means that Jenkins users install arbitrary code which cannot be easily verified by them. This code may include usage stats collection, various exploits, backdoors, etc. As a Jenkins community we cannot verify such plugins. Since the plugins have full access to Jenkins internals, closed-source dependencies are also a security concern for the project.

The topic about hosting plugins with closed source dependencies was reviewed by the Jenkins Governance meeting  on June 17, 2020 (meeting notes: https://docs.google.com/document/d/11Nr8QpqYgBiZjORplL_3Zkwys2qK1vEvK-NYyYa4rzg/edit#bookmark=id.u9aiz859uty4). Meeting participants voted unanimously against changing the policy and allowing plugins with closed source dependencies in the community-provided update centers.

I hope it explains the position of the Jenkins project. Again, there are solutions available to end users.

How to get the TFS plugin restored in the update centers?

https://github.com/microsoft/team-explorer-everywhere indeed looks promising, it has the OSI-compliant MIT license. At the same time there is no clear evidence that the consumed JAR represents the open source code. The library would need to be updated to a recent version hosted in a public repository like Maven Central. It looks to be doable, any contributions would be welcome.

Same for fixing the security issues and tabs-to-divs compatibility. Both these changes seem to be quite trivial so that they could be implemented without strong Jenkins developer expertise. I am ready to help with reviews if needed.

 

> MS may have simply forgotten (or can't be bothered) to get around to requesting it be dropped from the Jenkins plugin repo.

If Microsoft is unresponsive, we have a plugin adoption process in place: https://www.jenkins.io/doc/developer/plugin-governance/adopt-a-plugin/ . Any current plugin user can step up and request transfer of the plugin ownership. 

> there's no interest in having the plugin published, which is a problem in itself

As a Jenkins community, we are interested in having any plugin viable published. But not at the cost of the Jenkins users who rely on the Jenkins publishing policies. 

> The refusal of JenkinsSCI to update this means we are going to have to stay on older, insecure, versions of Jenkins because of a political pissing match. 

Casey May I remind you about Code of Conduct: https://www.jenkins.io/project/conduct/ . Apart from that, note that the Jenkins is a community and a project driven by contributors. We are not a corporation, and there is no "refuse" or "implement for others" as well as there is no SLA. If someone contributes a fix by submitting a pull request to the TFS plugin, we will do our best to help it landed. But there is no guarantee that someone will submit a fix for your needs. This is open-source.

> As a user of TFS for almost 8 years, I am not entirely surprised Microsoft and leah antkiewicz have not responded to your requests. It seems since TFS 2015 and the introduction of robust support for Git, MS have frozen development of the TVFC side of TFS. Since the acquisition of GitHub, they appear to have abandoned support as well. It would be nice if they would actually at least respond with some useful information.

I am not surprised as well. As a former and not so happy TFS user, I can only recommend contacting your support contacts if you are a paying customer. In a few cases it worked for us in early 2010s. I can ping my contacts at Microsoft to get a response, but I doubt one would see an active maintenance

 

 

 

 

[jenkins-infra-bot]

ianw:

I appreciate and respect Oleg Nenashev's comments. We all wish MS would be more responsive.

re: "https://github.com/microsoft/team-explorer-everywhere indeed looks promising, it has the OSI-compliant MIT license. At the same time there is no clear evidence that the consumed JAR represents the open source code. ",

tfs-plugin packages: com.microsoft.tfs.sdk-14.0.3.jar (SHA1: 8ED792570F1510CEDE1E7C44A46838850AA9A5AF).  That jar is available from Microsoft Download Center, choose TFS-SDK-14.0.3.zip. The jar contained within the zip has the same SHA1. The source for that jar is microsoft / team-explorer-everywhere. The source contains the MIT license. Internally, the jar contains Apache License Version 2.0, January 2004 and secondary licenses. Does that chain of evidence meets the threshold for Jenkins Governance for the license?

It would be preferable that the official plugin be fixed (including table-to-divs and security issue) rather than having 1000's users rebuild it themselves.

I don't believe MS have published the tfs-sdk.jar anywhere like Central or mvnrepository.com, so any updates would require a similar approach to existing ... download the latest release zip, extract the jar, check it in to tfs-plugin repo, update references, etc.

If a PR were to be submitted with update to the plugin as I note above (plus the other fixes), is that approach sufficient to meet Jenkins publishing policies acceptable to the Governance team and have the plugin reinstated ? OR must the tfs-sdk be by forked and released somewhere so it can be pulled from Central for the build?

Must someone also request adoption of the tfs-plugin, even though MS team have not formally abandoned it,so as to approve the PR and cut the release or is Oleg able/prepared to do? I see MS have provided formal notice for the Azure plugins, but stoney silence for TFS.

[jenkins-infra-bot]

oleg_nenashev:

> Must someone also request adoption of the tfs-plugin, even though MS team have not formally abandoned it,so as to approve the PR and cut the release or is Oleg able/prepared to do? I see MS have provided formal notice for the Azure plugins, but stoney silence for TFS.

Somebody would need to take ownership according to the current policy. I have an action item for an alternative way in my backlog, but I have not created a JEP for that yet. 

Plugin adoption process is specifically designed for the case when maintainers are unresponsive: https://www.jenkins.io/doc/developer/plugin-governance/adopt-a-plugin/ . If Microsoft representatives do not explicitly reject the request, we can transfer permissions after 2 weeks of waiting.

> Does that chain of evidence meets the threshold for Jenkins Governance for the license? 

To be discussed. "The source contains the MIT license. Internally, the jar contains Apache License Version 2.0, January 2004 and secondary licenses". License difference is still a concern. 

> must the tfs-sdk be by forked and released somewhere so it can be pulled from Central for the build?

There is no specific regarding the location of an open source library at the moment. There was a discussion about strictly requiring Maven Central or Jenkins Artifactory at the last contributor summit, but we have not implemented this policy yet. CC Wadeck Follonier Daniel Beck .

[jenkins-infra-bot]

mtughan:

Oleg Nenashev, thank you for your insightful comments. I completely agree with forbidding plugins with closed-source dependencies due to the possibility for arbitrary code being executed, so I don't have a problem with that policy at all. The TFS SDK appears to be a very unusual scenario in that the source itself appears to be public, but Microsoft (at least at first glance) does not appear to have published the code in any way that it can be automatically consumed by Maven or another build process into a project, which is unfortunate.

> Any current plugin user can step up and request transfer of the plugin ownership.
I am more than willing to put some time in to see about fixing the security issue and tables-to-divs, but wouldn't want to do so if my changes have no chance of being merged. I could also potentially see stepping in as a maintainer, even temporarily, but am having trouble finding exactly what would be expected of me if I were to request adoption. Is there a document anywhere that explains what the expectations are of a plugin maintainer?

[jenkins-infra-bot]

oleg_nenashev:

Hi Michael Tughan

> Is there a document anywhere that explains what the expectations are of a plugin maintainer?

Good question. Actually we do not have such a document, and it would be great to have one. I will make sure we add it to https://www.jenkins.io/doc/developer/plugin-governance/adopt-a-plugin/#faq 

Historically, our approach has been "A maintainer is better than no maintainer". We do not require plugin maintainers to commit to anything when they send the adoption request, though we ask them to declare their intent. But there is no SLA of any kind.  A healthy plugin state would require some time from the maintainer to review the incoming pull requests, triage issue reports and to make releases when there enough changes. Also, sometimes the Jenkins Security Officer (Daniel Beck at the moment) contacts maintainer to notify them about the security issue and to coordinate the fix/advisory. The fix itself might be done by a maintainer or by another security team member. All of these activities require some time, but not that much when the plugin does not evolve quickly. I'd guess it would be a case for the TFS plugin which should just work without need to add new features.

Just to share my experience, at some point I used to "maintain" around 30 plugins in the life support mode. It was taking several hours per week. So in my experience any single plugin was not a real burden (maybe except EnvInject plugin)

 

 

[jenkins-infra-bot]

ianw:

I did some digging and found the last time someone was added by MS as a maintainer, it took 3 permissions:
Adding leantk to the maintainer list #1396, so a new maintainer(s) should similarly request the trio.

There's no mention in the instructions about updating the embedded tfs-sdk. If embedding the jar is eventually Ok'd, an explanatory ./tfs-sdk/README.md would be helpful.

[jenkins-infra-bot]

ianw:

Oleg Nenashev, some more digging reveals this Closed issue for tfs-sdk: [Question] Included Team Services SDK for Java - licensed under MIT?. David Staheli (aka davidstaheli) is a maintainer for both TEE/tfs-sdk and tfs-plugin and a Microsoft employee. Is that sufficient to address the license question or must they administratively clean it up? How to proceed?

[jenkins-infra-bot]

oleg_nenashev:

Ian Williams as long as JAR content are fine and licensed properly, we can live with the deviation in the repository (IMHO). Would be great to get feedback from others

 

 

[jenkins-infra-bot]

danielbeck:

IMO the license issue is resolved now. Thanks for the investigation.

In terms of maintainership, we can add maintainers to unmaintained plugins, and given the complete lack of responses from MS folks both here and in the security tracker is indication enough of that IMO.

In regards to security issues, we just announced more problems with the plugin in https://www.jenkins.io/security/advisory/2021-03-30/ and we require those be addressed before the plugin distribution is restored. I'd be happy to be involved in review to ensure those are properly resolved.