Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Windows Defender issues #9

Open
PyvesB opened this issue Nov 20, 2020 · 38 comments
Open

Fix Windows Defender issues #9

PyvesB opened this issue Nov 20, 2020 · 38 comments
Labels
bug Something isn't working windows

Comments

@PyvesB
Copy link

PyvesB commented Nov 20, 2020

Hello, thanks for creating this repository! 👋🏻

Describe the bug

Eclipse, IntelliJ and other Java programs are significantly slowed down by Windows Defender. This makes Java development on Windows difficult: for example, an IDE may take several minutes to launch compared to a few seconds on other operating systems.

This issue was originally reported on the Microsoft Feedback Hub and initially received many upvotes, even though the voting system has since then been removed from the hub. Here's the full report written by Rolf T:

Windows Defender significantly slows down access to jar files, even if these jars are correctly signed. As a result, any java process is significantly slowed down. This is especially true for IDEs such as Eclipse and IntelliJ which regularly access jar files while they are being used. This results in many random hiccups and slow downs of the processes, up to the point where they become unusable.
The only viable workaround seems to exclude these processes from being scanned by Windows Defender, which kind of defies the whole purpose of having the scans at all.

Just only running "jar -tf any.jar" makes Windows Defender spike on CPU usage.

See also:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=548443
https://blog.jetbrains.com/idea/2019/06/intellij-idea-2019-2-eap5-commit-from-local-changes-and-more/

The problem becomes worse on a laptop while on a battery power. To reduce battery consumption, the priority of Windows Defender seems to be lowered. As a result, the slowdown of the processes accessing a jar file is even worse.

To Reproduce

Install and launch Eclipse, IntelliJ or other Java programs on Windows.

Expected behavior

Windows Defender should not slow down Java programs, especially if these are signed properly.

Additional context

Some programs provide utilities to automatically add themselves to the Windows Defender exclusion lists and tamper with the antivirus's settings. This feels wrong and makes the Java ecosystem on Windows less secure.

@vogella
Copy link

vogella commented May 6, 2021

Any progress here? Eclipse IDE is deadly slow on Windows with virus scanner active.

@microsoft microsoft deleted a comment from geartech Jul 27, 2021
@Phillipus
Copy link

+1

Excluding my Eclipse folder and workspace from Windows Defender gives me a massive speed boost for Eclipse startup time.

@laeubi
Copy link

laeubi commented Apr 4, 2022

Excluding my Eclipse folder and workspace from Windows Defender gives me a massive speed boost for Eclipse startup time.

I'm waiting for the first maleware that aims at eclipse installation folders in the hope they are excluded from anti-virus/maleware scans... 👎

@merks
Copy link

merks commented Apr 4, 2022

Re eclipse-platform/eclipse.platform.swt#10 (comment)

I started to write the following, but I later decided to keep the request focused on the tree selection issue:

image

One can easily see here that more time is spent "scanning" than in the actual process doing real work. And all the scanning seems to happen in the installer when creating the new installation and then each and every time any installation is launched....

Given this issue is now link to the tree selection issue, perhaps the powers that be looking at either issue will see both issues...

@rolfth
Copy link

rolfth commented Apr 4, 2022

IntelliJ suffers too. It has automatic detection for Windows Defender and offers to exclude directories:
https://intellij-support.jetbrains.com/hc/en-us/articles/360006298560

@kjsmita6
Copy link

kjsmita6 commented Apr 4, 2022

Using maven-wagon-plugin:merge-maven-repos from Eclipse to merge Maven repositories is slowed down about 20x with defender on. I would expect similar slowdowns with other processes that move files around.

Plugin info: https://www.mojohaus.org/wagon-maven-plugin/merge-maven-repos-mojo.html

The jar files are already downloaded on the computer - why do they need to be scanned when they are copied from one location to another on the same computer which downloaded them? Setting an exclusion for *.jar files has significantly sped it up.
procplot_wdavdaemon_unpriviledged
Graph of defender CPU usage during a typical development day (span of about 2.5 hours). CPU peaks around 800% and reaches 300% often. I did not run a scan during any of these times - it was just in the background.

Edit: This is happening on Mac.

@vogella
Copy link

vogella commented Jun 16, 2022

cc @stephenrwalli can you help to address this, this makes the Eclipse IDE very slow on Windows.

@karianna
Copy link
Member

CC @brunoborges and @gdams - I know we've been able to talk to the Defender Team about this before and it's a bit of whack-a-mole with getting exceptions and then newer versions perhaps overriding that. Can we engage on a more permanent fix?

@brunoborges
Copy link
Member

There is an ongoing effort about this, in general. Not just for Java. Defender design impacts all developer tools and languages.

I don't have more info to share at the moment, though.

But rest assured, Windows team is looking into this.

@TIBCOeddie
Copy link

Any chance for an end-of-year Defender team update?

@marioja
Copy link

marioja commented Mar 15, 2023

@brunoborges Hi Bruno. In July last year you said that the Windows team is looking into this. Can you tell the java development community:

  1. what is the status of this issue with defender?
  2. what is the priority Microsoft is giving to this issue?
  3. is there an estimate to when this would be fixed?
    I know it is hard to share the information I am asking but look at it from the other end, all developers are affected. Thanks.

@brunoborges
Copy link
Member

Hey @marioja , thanks for pinging me here.

I will meet with the Windows team in the coming weeks and I'll bring this topic up again witblh them.

Stay tuned!

@ecki
Copy link

ecki commented Mar 22, 2023

Just to mention it from my duplicate report, it’s not only an issue with defender, in fact other malware scanners are partially even worse (MS can’t fix them but at least influence with certification criterias). Some of the issues I observed with trendmicro:

  • behavioral scanner exclusion is based on the binary, but excluding java.exe is of course very bad for security as this is the first LoL/host for an attacker then
  • Some scanners scan on each read/open of zip/jar instead of caching their result or only scanning on writes
  • Scanners keeping file deletes delayed are very bad for maven atomic rename stability (at least until OpenJDK fixes the MOVEFILE_WRITETHROUGH (https://mail.openjdk.org/pipermail/nio-dev/2023-March/013270.html)

@michael-o
Copy link

@ecki You found my report, this is truly a pain.

@ecki
Copy link

ecki commented Mar 23, 2023

@ecki You found my report, this is truly a pain.

I see you ,)

but more seriously, I just noticed your report is probably only related to dirty buffers on windows, not the busy Filmhandle by malware scanners (but I have seen those effects, too)

@brunoborges
Copy link
Member

@michael-o have you noticed this issue impacting some of the Apache big data projects?

@michael-o
Copy link

@michael-o have you noticed this issue impacting some of the Apache big data projects?

I am not involved in Big Data, in Maven only.

@brunoborges
Copy link
Member

@michael-o would you have a chance to suggest a Maven project we can investigate to track the performance?

@bendem
Copy link

bendem commented May 17, 2023

We are being hit by this running tomcat on windows server. The first page load is excruciatingly slow because of defender scanning all jars on open.

I'm sure the team has already thought about this and it's not that simple (it never is), but all jars on maven central are supposed to be signed. Could it be somehow possible to trust those signature which are easy to untrust with a definition update? That would reduce the amount of full jar scan to a minimum since I'd guess 80% of jars to scan come from maven central.

Here is an excerpt from a trace I ran during a cpu spike, tomcat jars are pretty clear offenders:
1    346,4064ms  C:\tomcat9\WEB-INF\lib\opensaml-xacml-api-3.3.0.jar
1    308,6898ms  C:\tomcat9\WEB-INF\lib\jsr311-api-1.1.1.jar
1    257,4115ms  C:\tomcat9\WEB-INF\lib\wss4j-bindings-2.2.3.jar
1    240,6490ms  C:\tomcat9\WEB-INF\lib\stax-api-1.0.1.jar
1    237,6721ms  C:\tomcat9\WEB-INF\lib\slf4j-api-1.7.36.jar
1    230,4865ms  C:\tomcat9\WEB-INF\lib\swagger-annotations-1.5.21.jar
1    200,1608ms  C:\tomcat9\WEB-INF\lib\swisscom-sender-14.4.0.1.jar
1    192,8432ms  C:\tomcat9\WEB-INF\lib\opensaml-xacml-saml-impl-3.3.0.jar
1    172,7117ms  C:\tomcat9\WEB-INF\lib\log4j-web-2.17.2.jar
1    171,9641ms  C:\tomcat9\WEB-INF\lib\saaj-api-1.3.jar
1    169,7886ms  C:\tomcat9\WEB-INF\lib\jstl-api-1.2.jar
1    167,6611ms  C:\tomcat9\WEB-INF\lib\opensaml-profile-api-3.3.0.jar
1    159,3315ms  C:\tomcat9\WEB-INF\lib\odmg-3.0.jar
1    156,3027ms  C:\tomcat9\WEB-INF\lib\relaxng-datatype-2.3.7.jar
1    146,7319ms  C:\tomcat9\WEB-INF\lib\spring-security-cas-5.7.4.jar
1    135,4048ms  C:\tomcat9\WEB-INF\lib\ntlmv2-lib-1.0.5.jar
1    129,4833ms  C:\tomcat9\WEB-INF\lib\smsSender-connector-ifaces-14.4.0.1.jar
1    128,0774ms  C:\tomcat9\WEB-INF\lib\spring-jcl-5.3.23.jar
1    121,5627ms  C:\tomcat9\WEB-INF\lib\spring-security-kerberos-core-1.0.1.RELEASE.jar
1    108,6158ms  C:\tomcat9\WEB-INF\lib\smsSender-dao-14.4.0.1.jar
1    108,5201ms  C:\tomcat9\WEB-INF\lib\smsSender-webservices-14.4.0.1.jar
1    106,1685ms  C:\tomcat9\WEB-INF\lib\jta-1.1.jar
1     90,3936ms  C:\tomcat9\WEB-INF\lib\opencsv-2.3.jar
1     86,4539ms  C:\tomcat9\WEB-INF\lib\spring-security-taglibs-5.7.4.jar
1     65,3030ms  C:\tomcat9\WEB-INF\lib\xpp3_min-1.1.3.4.O.jar
1     64,4819ms  C:\tomcat9\WEB-INF\lib\opensaml-xacml-saml-api-3.3.0.jar
1     62,1405ms  C:\tomcat9\WEB-INF\lib\spring-security-kerberos-client-1.0.1.RELEASE.jar
1     58,4727ms  C:\tomcat9\WEB-INF\lib\ntlmv2-filter-1.0.5.jar
1     27,4033ms  C:\tomcat9\WEB-INF\lib\ordermanager-webservices-1.0.0.jar
1     25,8181ms  C:\tomcat9\WEB-INF\lib\opensaml-xacml-impl-3.3.0.jar
1     18,7982ms  C:\tomcat9\WEB-INF\lib\opensaml-xmlsec-impl-3.3.0.jar
1     18,4236ms  C:\tomcat9\WEB-INF\lib\opensaml-saml-impl-3.3.0.jar
1     17,9277ms  C:\tomcat9\WEB-INF\lib\ojdbc5-11.2.0.4.jar
1     16,8699ms  C:\tomcat9\WEB-INF\lib\spring-web-5.3.23.jar
1     15,4870ms  C:\tomcat9\WEB-INF\lib\saaj-impl-1.3.2.jar
1     15,3993ms  C:\tomcat9\WEB-INF\lib\mail-1.4.1.jar
1     15,2199ms  C:\tomcat9\WEB-INF\lib\serializer-2.7.1.jar
1     14,9561ms  C:\tomcat9\WEB-INF\lib\snakeyaml-1.30.jar
1     14,8675ms  C:\tomcat9\WEB-INF\lib\spring-tx-5.3.23.jar
1     14,7814ms  C:\tomcat9\WEB-INF\lib\xmlschema-core-2.2.4.jar
1     14,7109ms  C:\tomcat9\WEB-INF\lib\spring-beans-5.3.23.jar
1     14,6513ms  C:\tomcat9\WEB-INF\lib\wss4j-policy-2.2.3.jar
1     14,6504ms  C:\tomcat9\WEB-INF\lib\log4j-api-2.17.2.jar
1     14,6358ms  C:\tomcat9\WEB-INF\lib\rngom-2.3.7.jar
1     14,4343ms  C:\tomcat9\WEB-INF\lib\spring-aop-5.3.23.jar
1     14,4105ms  C:\tomcat9\WEB-INF\lib\metrics-core-4.2.12.jar
1     14,0816ms  C:\tomcat9\WEB-INF\lib\mchange-commons-java-0.2.15.jar
1     14,0463ms  C:\tomcat9\WEB-INF\lib\opensaml-xmlsec-api-3.3.0.jar
1     14,0081ms  C:\tomcat9\WEB-INF\lib\xsom-2.3.7.jar
1     13,9623ms  C:\tomcat9\WEB-INF\lib\mysql-connector-j-8.0.31.jar
1     13,8973ms  C:\tomcat9\WEB-INF\lib\spring-expression-5.3.23.jar
1     13,8695ms  C:\tomcat9\WEB-INF\lib\swagger-core-1.5.21.jar
1     13,8637ms  C:\tomcat9\WEB-INF\lib\opensaml-saml-api-3.3.0.jar
1     13,7666ms  C:\tomcat9\WEB-INF\lib\spring-security-core-5.7.4.jar
1     13,7562ms  C:\tomcat9\WEB-INF\lib\spring-security-web-5.7.4.jar
1     13,7337ms  C:\tomcat9\WEB-INF\lib\spring-boot-2.7.5.jar
1     13,7016ms  C:\tomcat9\WEB-INF\lib\swagger-jaxrs-1.5.21.jar
1     13,6258ms  C:\tomcat9\WEB-INF\lib\swagger-models-1.5.21.jar
1     13,6084ms  C:\tomcat9\WEB-INF\lib\spring-webmvc-5.3.23.jar
1     13,5148ms  C:\tomcat9\WEB-INF\lib\spring-security-ldap-5.7.4.jar
1     13,4818ms  C:\tomcat9\WEB-INF\lib\jstl-impl-1.2.jar
1     13,4699ms  C:\tomcat9\WEB-INF\lib\velocity-1.5.jar
1     13,4645ms  C:\tomcat9\WEB-INF\lib\spring-jdbc-5.3.23.jar
1     13,4505ms  C:\tomcat9\WEB-INF\lib\servlet-api-2.5.jar
1     13,4129ms  C:\tomcat9\WEB-INF\lib\log4j-core-2.17.2.jar
1     13,4073ms  C:\tomcat9\WEB-INF\lib\tomcat-servlet-api-9.0.69.jar
1     13,3699ms  C:\tomcat9\WEB-INF\lib\stax2-api-3.1.4.jar
1     13,3469ms  C:\tomcat9\WEB-INF\lib\woodstox-core-asl-4.2.0.jar
1     13,3369ms  C:\tomcat9\WEB-INF\lib\smsSender-core-14.4.0.1.jar
1     13,3043ms  C:\tomcat9\WEB-INF\lib\spring-boot-autoconfigure-2.7.5.jar
1     13,2490ms  C:\tomcat9\WEB-INF\lib\spring-security-config-5.7.4.jar
1     13,2054ms  C:\tomcat9\WEB-INF\lib\opencsv-4.6.jar
1     12,9920ms  C:\tomcat9\WEB-INF\lib\neethi-3.0.3.jar
1     12,9455ms  C:\tomcat9\WEB-INF\lib\spring-context-5.3.23.jar
1     12,9134ms  C:\tomcat9\WEB-INF\lib\purejavacomm-1.0.2.RELEASE.jar
1     12,8529ms  C:\tomcat9\WEB-INF\lib\woodstox-core-5.0.3.jar
1     12,8248ms  C:\tomcat9\WEB-INF\lib\oro-2.0.8.jar
1     12,7502ms  C:\tomcat9\WEB-INF\lib\spring-orm-5.3.23.jar
1     12,7160ms  C:\tomcat9\WEB-INF\lib\tomcat-el-api-9.0.69.jar
1     12,6260ms  C:\tomcat9\WEB-INF\lib\spring-context-support-5.3.23.jar
1     12,5416ms  C:\tomcat9\WEB-INF\lib\opensaml-security-api-3.3.0.jar
1     12,5313ms  C:\tomcat9\WEB-INF\lib\xmlsec-2.1.2.jar
1     12,4802ms  C:\tomcat9\WEB-INF\lib\spring-core-5.3.23.jar
1     12,4397ms  C:\tomcat9\WEB-INF\lib\quartz-2.3.2.jar
1     12,3974ms  C:\tomcat9\WEB-INF\lib\spring-security-crypto-5.7.4.jar
1     12,3509ms  C:\tomcat9\WEB-INF\lib\poi-3.16-beta1.jar
1     12,3408ms  C:\tomcat9\WEB-INF\lib\spring-security-acl-5.7.4.jar
1     12,2917ms  C:\tomcat9\WEB-INF\lib\wss4j-ws-security-common-2.2.3.jar
1     12,2394ms  C:\tomcat9\WEB-INF\lib\spring-ldap-core-2.4.1.jar
1     12,1972ms  C:\tomcat9\WEB-INF\lib\spring-jms-5.3.23.jar
1     12,1442ms  C:\tomcat9\WEB-INF\lib\xml-apis-1.4.01.jar
1     12,1345ms  C:\tomcat9\WEB-INF\lib\wss4j-ws-security-stax-2.2.3.jar
1     12,0500ms  C:\tomcat9\WEB-INF\lib\poi-ooxml-3.16-beta1.jar
1     12,0128ms  C:\tomcat9\WEB-INF\lib\opensaml-soap-api-3.3.0.jar
1     11,9963ms  C:\tomcat9\WEB-INF\lib\opensaml-core-3.3.0.jar
1     11,9096ms  C:\tomcat9\WEB-INF\lib\txw2-2.3.7.jar

@ecki
Copy link

ecki commented May 17, 2023

The maven signatures are unfortunatelly detached, you don’t have them in the jar (and using jar signing would be possible but is frowned upon).

in case of antivirus it would be an option to scan files only when they are written and then attach a signature/version information (NTFS stream or extended attributes) so you can verify the local filesystem is unaltered and the scan was with an up-to-date scanner. In case of new malware signatures the background scanner could refresh those markers.

@michael-o
Copy link

@michael-o would you have a chance to suggest a Maven project we can investigate to track the performance?

None in particular.

@michael-o
Copy link

michael-o commented May 17, 2023

We are being hit by this running tomcat on windows server. The first page load is excruciatingly slow because of defender scanning all jars on open.

I'm sure the team has already thought about this and it's not that simple (it never is), but all jars on maven central are supposed to be signed. Could it be somehow possible to trust those signature which are easy to untrust with a definition update? That would reduce the amount of full jar scan to a minimum since I'd guess 80% of jars to scan come from maven central.
Here is an excerpt from a trace I ran during a cpu spike, tomcat jars are pretty clear offenders:

<rant>
Well, there is a solution to that problem: Use an operating system which what YOU want and not what OTHERS want.
</rant>

@bendem
Copy link

bendem commented May 19, 2023

<rant>
Well, there is a solution to that problem: Use an operating system which what YOU want and not what OTHERS want.
</rant>

Please, do provide us with the millions required to run a municipality and I'll happily be able to chose what OS I run my software on. Otherwise, maybe refrain from derailing the conversation into wasted heat.

@michael-o
Copy link

<rant>
Well, there is a solution to that problem: Use an operating system which what YOU want and not what OTHERS want.
</rant>

Please, do provide us with the millions required to run a municipality and I'll happily be able to chose what OS I run my software on. Otherwise, maybe refrain from derailing the conversation into wasted heat.

I wasn't able to hold my self off. But honestly, this has been open now for almost three years. Do you really expect anything to happen here?

@ecki
Copy link

ecki commented May 23, 2023

@michael-o do not give up, just read that in the news:

Microsoft has also created Dev Drive inside Dev Home, a new storage volume that’s customized for 
developers and uses Microsoft’s latest Resilient File System (ReFS) alongside a performance mode
for Microsoft Defender that will improve build times for heavy I/O operations by up to 30 percent.
“The new performance mode is more secure for your workloads than folder or process exclusions,
providing an ultimate solution to balance security with performance,”

@ecki
Copy link

ecki commented May 23, 2023

@michael-o would you have a chance to suggest a Maven project we can investigate to track the performance?

If you want to publish some numbers Apache KAraf OSGi is reasonbable big and still pretty selfcontained. We use that internally as a benchmark quite often (but then again, we depend on it).

@michael-o
Copy link

@michael-o do not give up, just read that in the news:

Microsoft has also created Dev Drive inside Dev Home, a new storage volume that’s customized for 
developers and uses Microsoft’s latest Resilient File System (ReFS) alongside a performance mode
for Microsoft Defender that will improve build times for heavy I/O operations by up to 30 percent.
“The new performance mode is more secure for your workloads than folder or process exclusions,
providing an ultimate solution to balance security with performance,”

Sounds like a totally convoluted solution to a simple problem: Don't scan.

@ecki
Copy link

ecki commented May 23, 2023

Sounds like a totally convoluted solution to a simple problem: Don't scan.

Well yes, but for those who dont want to scan the solution is easy: dont scan :)

@brunoborges
Copy link
Member

Sounds like a totally convoluted solution to a simple problem: Don't scan.

The scanning should be transparent. Experienced developers will always have the option to disable scanning (unless forbidden by IT Corp rules). But for others, it is better to leverage the Dev Home experience. Faster and yet still secure.

@karianna
Copy link
Member

karianna commented Jun 6, 2023

@michael-o
Copy link

michael-o commented Jun 6, 2023

@rolfth
Copy link

rolfth commented Jun 7, 2023

IMHO, the development mode doesn't solve the reported problem. It is not a problem for developers, but a problem when running Java programs. It is a generic end-user problem and it only affects developers because they happen to be heavy end-users.

The problem is that Windows Defender doesn't trust jar-files. Therefore, it scans the full content of a jar-file upon access. This takes a lot of time and resources (battery drain). On top of that, scan results are not cached, which results that this scanning overheard is observed each time a application starts. This results in poor performance of every Java application on a platform with Windows Defender running.
Meanwhile, the Java community is making great effort to digitally sign jars, such that content could be trusted without scanning the content. These efforts are ignored by the Windows Defender team.

Apparently, the problem and its impact is not understood by Microsoft. The issue is affecting thousands of applications, millions of people and takes many hours of productive time.

@brunoborges
Copy link
Member

Defender doesn't trust any ZIP file. A JAR file is a ZIP file, and WD will treat as such.

I am not sure about scanning results being scanned, though. Sure that would help.

@danielreuther
Copy link

Defender doesn't trust any ZIP file. A JAR file is a ZIP file, and WD will treat as such.

I understand that argument, however, as @rolfth pointed out, signatures could help WD distinguish between a simple ZIP file and a potentially more trustworthy signed JAR file.

@ecki
Copy link

ecki commented Jun 7, 2023

I would not waste time with jar signatures, they are very uncommon (and you can as usual easily sign malware as well). Not to mention that signature checking is slow as well.

@ecki
Copy link

ecki commented Jun 7, 2023

IMHO, the development mode doesn't solve the reported problem.

Depends on which report :) the initial report in this ticket is about IDE and Build systems, for that development mode with trusted filesystem helps (for those who can use it), for the also mentioned application startup issues it won’t help - but those are less severe imho.

@rolfth
Copy link

rolfth commented Jun 9, 2023

IMHO, the development mode doesn't solve the reported problem.

Depends on which report :) the initial report in this ticket is about IDE and Build systems, for that development mode with trusted filesystem helps (for those who can use it), for the also mentioned application startup issues it won’t help - but those are less severe imho.

Yes, this ticket is reports the effect on development efficiency. However, it starts with a generic statement. Also it references to the original issue that I reported on Microsoft Feedback Hub, which is generic for Java applications too.

@marschall
Copy link

I would not waste time with jar signatures, they are very uncommon (and you can as usual easily sign malware as well). Not to mention that signature checking is slow as well.

I'm old enough to remember NGSCB, Palladium, Trustworthy Computing where Microsoft argued signed code = safe.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working windows
Projects
None yet
Development

No branches or pull requests