MonoGame Pipeline Tool Custom Processor bug #6443

Quinn-L · 2018-09-26T12:34:36Z

I have an issue with MonoGame Pipeline Tool 3.7.0.1708 where my custom processor project is no longer accepted by the pipeline tool and it gives a "Invalid / Missing Processor" where it used to be assigned.
Doing some code digging, it looks like it occurred due to commit #6202, where the following code is throwing an exception (which is swallowed) in the file PipelineTypes.cs:

  var a = Assembly.Load(File.ReadAllBytes(path));      // This line is the problem
  var types = a.GetTypes();                            // This throws an exception

Looking at the previous code, it used to be var a = Assembly.LoadFrom(path);

Fishing in stackoverflow it seems there are subtle differences between Assembly.Load and Assembly.LoadFrom

I don't fully understand what's going on, however changing this line back makes the pipeline recognize my project again.

FYI, the exception that got thrown is:
System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.

What version of MonoGame does the bug occur on:

MonoGame Pipeline Tool 3.7.0.1708

What operating system are you using:

Windows

EDIT:
Looking a bit more carefully at #6202, I see the point is so the custom pipeline dll isn't locked by the pipeline tool, which is why Assembly.Load is used instead of Assembly.LoadFrom.
It seems the issue with Assembly.Load(byte[]) is that it can't locate the dependent libraries because it no longer knows its location of itself so the app only looks in the pipeline app's directory?
In any case, changing the code to include AppDomain.CurrentDomain.AssemblyResolve & loading dependencies:

Dictionary<string, Assembly> asmMap = new Dictionary<string, Assembly>();
var mainAsm = AppDomain.CurrentDomain.GetAssemblies();
ResolveEventHandler resolveHandler = (object sender, ResolveEventArgs args) =>
{
    asmMap.TryGetValue(args.Name, out var assembly);
    return assembly;
};
AppDomain.CurrentDomain.AssemblyResolve += resolveHandler;

foreach (var path in assemblyPaths)
{
    try
    {
        var a = Assembly.Load(File.ReadAllBytes(path));
        asmMap[a.FullName] = a;
        foreach (var childAsmName in a.GetReferencedAssemblies())
        {
            var aa = mainAsm.FirstOrDefault(x => x.FullName == childAsmName.FullName);
            if (aa != null)
            {
                continue;
            }

            var chPath = Path.Combine(Path.GetDirectoryName(path), childAsmName.Name + ".dll");
            if (!File.Exists(chPath))
            {
                continue;
            }
            var a2 = Assembly.Load(File.ReadAllBytes(chPath));
            asmMap[a2.FullName] = a2;
        }
        var types = a.GetTypes();
        ProcessTypes(types);

        var watch = new FileSystemWatcher();
        watch.Path = Path.GetDirectoryName(path);
        watch.EnableRaisingEvents = true;
        watch.Filter = Path.GetFileName(path);
        watch.Changed += (sender, e) =>
        {
            if (Path.GetFileName(path) == e.Name)
                PipelineController.Instance.OnReferencesModified();
        };
        watch.Created += (sender, e) =>
        {
            if (Path.GetFileName(path) == e.Name)
                PipelineController.Instance.OnReferencesModified();
        };

        _watchers.Add(watch);
    }
    catch (Exception e)
    {
        //Logger.LogWarning(null, null, "Failed to load assembly '{0}': {1}", assemblyPath, e.Message);
        // The assembly failed to load... nothing
        // we can do but ignore it.
        continue;
    }
}

AppDomain.CurrentDomain.AssemblyResolve -= resolveHandler;

I'm not an expert in this area so I don't know if assuming dependent dlls being in the same folder as the desired assembly is enough (it is sufficient for my project...).

Furthermore, PropertyGridTable.Update() needed to be changed from
drawable.Invalidate();
to
Application.Instance.Invoke(() => drawable.Invalidate());
because when the file change detection occurs, it doesn't seem to be on the main UI thread and crashes the pipeline app.

The text was updated successfully, but these errors were encountered:

harry-cpp · 2018-09-28T11:25:26Z

System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.

Seems like your assembly is dependent on another assembly thats next to it, hmm... I'm looking into fixing this now.

harry-cpp · 2018-09-28T11:58:31Z

If you have a moment of time, please try Pipeline Tool from: http://teamcity.monogame.net/repository/download/MonoGame_PackagingWindows/71511:id/MonoGameSetup.exe?guest=1

Quinn-L · 2018-09-29T00:57:22Z

Thanks @cra0zy. You are correct that the assembly is dependent on another assembly in the same directory it lives in.
The link you've given works and picks up my processor again. 👍

Jjagg · 2018-09-29T14:32:09Z

Fixed in #6450!

harry-cpp · 2018-09-29T14:42:36Z

@Jjagg You don't need to mention that it was fixed in a PR as GitHub already said it.

Also on a related note, don't use rebase and merge unless you have to, stuff just gets more messy with who did what:

harry-cpp · 2018-09-29T14:50:14Z

*Squash and merge

Jjagg · 2018-09-29T17:29:36Z

I refer to the PR in a comment so there's a link to it. You linked to this issue in a commit, not in the PR itself.

harry-cpp · 2018-09-29T17:38:38Z

@Jjagg Oh, did not notice that, when you merge stuff normally you get a link back, see #6397, 1 more point for just merging stuff and not rebasing/squashing it :P

picnic · 2018-10-14T08:41:44Z

Hello, I still have the bug on MacOs, using the latest Monogame & Content Pipeline version on TeamCity (3.8.0.76). I attached a minimal project that leads to the error.
testxml.zip
Here is the error:

RunContentBuilder cible :
    /Library/Frameworks/Mono.framework/Versions/Current/bin/mono "/Library/Frameworks/Mono.framework/External/xbuild/MonoGame/v3.0/Tools/MGCB.exe" /quiet /platform:DesktopGL /@:"/Users/picnic/Projects/testxml/testxml/Content/Content.mgcb" /outputDir:"bin/DesktopGL/Content" /intermediateDir:"obj/DesktopGL/Content"
    /Users/picnic/Projects/testxml/testxml/Content/File.xml
    EXEC : warning : Exception of type 'System.Reflection.ReflectionTypeLoadException' was thrown.
    /Users/picnic/Projects/testxml/testxml/Content/File.xml : error : Importer 'XmlImporter' had unexpected failure!
    
    Unhandled Exception:
    System.IO.DirectoryNotFoundException: Could not find a part of the path "/Users/picnic/Projects/testxml/testxml/Content/obj/DesktopGL/Content/.mgstats".
      at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.Int32 bufferSize, System.Boolean anonymous, System.IO.FileOptions options) [0x00164] in <71f4e3d7d65e42b0a699d02e59766d3c>:0 
      at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.Int32 bufferSize, System.IO.FileOptions options, System.String msgPath, System.Boolean bFromProxy, System.Boolean useLongPath, System.Boolean checkHost) [0x00000] in <71f4e3d7d65e42b0a699d02e59766d3c>:0 
      at (wrapper remoting-invoke-with-check) System.IO.FileStream..ctor(string,System.IO.FileMode,System.IO.FileAccess,System.IO.FileShare,int,System.IO.FileOptions,string,bool,bool,bool)
      at System.IO.StreamWriter.CreateFile (System.String path, System.Boolean append, System.Boolean checkHost) [0x0001c] in <71f4e3d7d65e42b0a699d02e59766d3c>:0 
      at System.IO.StreamWriter..ctor (System.String path, System.Boolean append, System.Text.Encoding encoding, System.Int32 bufferSize, System.Boolean checkHost) [0x00055] in <71f4e3d7d65e42b0a699d02e59766d3c>:0 
      at System.IO.StreamWriter..ctor (System.String path, System.Boolean append, System.Text.Encoding encoding, System.Int32 bufferSize) [0x00000] in <71f4e3d7d65e42b0a699d02e59766d3c>:0 
      at System.IO.StreamWriter..ctor (System.String path, System.Boolean append, System.Text.Encoding encoding) [0x00000] in <71f4e3d7d65e42b0a699d02e59766d3c>:0 
      at (wrapper remoting-invoke-with-check) System.IO.StreamWriter..ctor(string,bool,System.Text.Encoding)
      at Microsoft.Xna.Framework.Content.Pipeline.ContentStatsCollection.Write (System.String outputPath) [0x00012] in <227b2d909358404f981099d80e2439b8>:0 
      at MGCB.BuildContent.Build (System.Int32& successCount, System.Int32& errorCount) [0x006df] in <764b3cbebab042cf8276bccb48963fef>:0 
      at MGCB.Program.Main (System.String[] args) [0x00089] in <764b3cbebab042cf8276bccb48963fef>:0 
    [ERROR] FATAL UNHANDLED EXCEPTION: System.IO.DirectoryNotFoundException: Could not find a part of the path "/Users/picnic/Projects/testxml/testxml/Content/obj/DesktopGL/Content/.mgstats".
      at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.Int32 bufferSize, System.Boolean anonymous, System.IO.FileOptions options) [0x00164] in <71f4e3d7d65e42b0a699d02e59766d3c>:0 
      at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.Int32 bufferSize, System.IO.FileOptions options, System.String msgPath, System.Boolean bFromProxy, System.Boolean useLongPath, System.Boolean checkHost) [0x00000] in <71f4e3d7d65e42b0a699d02e59766d3c>:0 
      at (wrapper remoting-invoke-with-check) System.IO.FileStream..ctor(string,System.IO.FileMode,System.IO.FileAccess,System.IO.FileShare,int,System.IO.FileOptions,string,bool,bool,bool)
      at System.IO.StreamWriter.CreateFile (System.String path, System.Boolean append, System.Boolean checkHost) [0x0001c] in <71f4e3d7d65e42b0a699d02e59766d3c>:0 
      at System.IO.StreamWriter..ctor (System.String path, System.Boolean append, System.Text.Encoding encoding, System.Int32 bufferSize, System.Boolean checkHost) [0x00055] in <71f4e3d7d65e42b0a699d02e59766d3c>:0 
      at System.IO.StreamWriter..ctor (System.String path, System.Boolean append, System.Text.Encoding encoding, System.Int32 bufferSize) [0x00000] in <71f4e3d7d65e42b0a699d02e59766d3c>:0 
      at System.IO.StreamWriter..ctor (System.String path, System.Boolean append, System.Text.Encoding encoding) [0x00000] in <71f4e3d7d65e42b0a699d02e59766d3c>:0 
      at (wrapper remoting-invoke-with-check) System.IO.StreamWriter..ctor(string,bool,System.Text.Encoding)
      at Microsoft.Xna.Framework.Content.Pipeline.ContentStatsCollection.Write (System.String outputPath) [0x00012] in <227b2d909358404f981099d80e2439b8>:0 
      at MGCB.BuildContent.Build (System.Int32& successCount, System.Int32& errorCount) [0x006df] in <764b3cbebab042cf8276bccb48963fef>:0 
      at MGCB.Program.Main (System.String[] args) [0x00089] in <764b3cbebab042cf8276bccb48963fef>:0 
    System.Reflection.ReflectionTypeLoadException: Exception of type 'System.Reflection.ReflectionTypeLoadException' was thrown.
      at (wrapper managed-to-native) System.Reflection.Assembly.GetTypes(System.Reflection.Assembly,bool)
      at System.Reflection.Assembly.GetTypes () [0x00000] in <71f4e3d7d65e42b0a699d02e59766d3c>:0 
      at Microsoft.Xna.Framework.Content.Pipeline.Serialization.Intermediate.IntermediateSerializer+<>c.<FindType>b__15_0 (System.Reflection.Assembly assembly) [0x00000] in <227b2d909358404f981099d80e2439b8>:0 
      at System.Linq.Enumerable+<SelectManyIterator>d__167`3[TSource,TCollection,TResult].MoveNext () [0x0004e] in <cc3b329d40bd4675ae0e985e302972af>:0 
      at System.Linq.Enumerable+WhereSelectEnumerableIterator`2[TSource,TResult].MoveNext () [0x00059] in <cc3b329d40bd4675ae0e985e302972af>:0 
      at System.Linq.Enumerable.TryGetFirst[TSource] (System.Collections.Generic.IEnumerable`1[T] source, System.Boolean& found) [0x00045] in <cc3b329d40bd4675ae0e985e302972af>:0 
      at System.Linq.Enumerable.FirstOrDefault[TSource] (System.Collections.Generic.IEnumerable`1[T] source) [0x00000] in <cc3b329d40bd4675ae0e985e302972af>:0 
      at Microsoft.Xna.Framework.Content.Pipeline.Serialization.Intermediate.IntermediateSerializer.FindType (System.String typeName) [0x00202] in <227b2d909358404f981099d80e2439b8>:0 
      at Microsoft.Xna.Framework.Content.Pipeline.Serialization.Intermediate.IntermediateReader.ReadTypeName () [0x00012] in <227b2d909358404f981099d80e2439b8>:0 
      at Microsoft.Xna.Framework.Content.Pipeline.Serialization.Intermediate.IntermediateReader.ReadObject[T] (Microsoft.Xna.Framework.Content.ContentSerializerAttribute format, Microsoft.Xna.Framework.Content.Pipeline.Serialization.Intermediate.ContentTypeSerializer typeSerializer, T existingInstance) [0x0009f] in <227b2d909358404f981099d80e2439b8>:0 
      at Microsoft.Xna.Framework.Content.Pipeline.Serialization.Intermediate.IntermediateReader.ReadObject[T] (Microsoft.Xna.Framework.Content.ContentSerializerAttribute format) [0x00017] in <227b2d909358404f981099d80e2439b8>:0 
      at Microsoft.Xna.Framework.Content.Pipeline.Serialization.Intermediate.IntermediateSerializer.Deserialize[T] (System.Xml.XmlReader input, System.String referenceRelocationPath) [0x00053] in <227b2d909358404f981099d80e2439b8>:0 
      at Microsoft.Xna.Framework.Content.Pipeline.XmlImporter.Import (System.String filename, Microsoft.Xna.Framework.Content.Pipeline.ContentImporterContext context) [0x00007] in <227b2d909358404f981099d80e2439b8>:0 
      at Microsoft.Xna.Framework.Content.Pipeline.ContentImporter`1[T].Microsoft.Xna.Framework.Content.Pipeline.IContentImporter.Import (System.String filename, Microsoft.Xna.Framework.Content.Pipeline.ContentImporterContext context) [0x0001c] in <227b2d909358404f981099d80e2439b8>:0 
      at MonoGame.Framework.Content.Pipeline.Builder.PipelineManager.ProcessContent (MonoGame.Framework.Content.Pipeline.Builder.PipelineBuildEvent pipelineEvent) [0x00079] in <227b2d909358404f981099d80e2439b8>:0 
    /Library/Frameworks/Mono.framework/External/xbuild/MonoGame/v3.0/MonoGame.Content.Builder.targets(84,5): error MSB3073: La commande "/Library/Frameworks/Mono.framework/Versions/Current/bin/mono "/Library/Frameworks/Mono.framework/External/xbuild/MonoGame/v3.0/Tools/MGCB.exe" /quiet /platform:DesktopGL /@:"/Users/picnic/Projects/testxml/testxml/Content/Content.mgcb" /outputDir:"bin/DesktopGL/Content" /intermediateDir:"obj/DesktopGL/Content"" s'est arrêtée avec le code 1.
Génération de la cible "RunContentBuilder" terminée dans le projet "testxml.MacOS.csproj" -- ÉCHEC.

harry-cpp · 2018-10-14T09:12:57Z

That error log has nothing to do with this error from what I can see, open up a separate issue about it.

rmhopkins · 2018-12-04T18:45:23Z

Am I seeing the same issue here? This is when trying to reference the DataTypes.dll in the Content Pipeline tool that comes with the latest GeonBit.UI nuget.

Visual Studio Enterprise 15.9.3
Monogame 3.7
GeonBit.UI 3.3.0

The Content Pipeline tool crashes with the following:

System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.
at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module)
at System.Reflection.Assembly.GetTypes()
at MonoGame.Tools.Pipeline.PipelineTypes.ResolveAssemblies(IEnumerable`1 assemblyPaths)
at MonoGame.Tools.Pipeline.PipelineTypes.Load(PipelineProject project)
at MonoGame.Tools.Pipeline.PipelineController.ResolveTypes()
at MonoGame.Tools.Pipeline.PipelineController.OnReferencesModified()
at MonoGame.Tools.Pipeline.CellRefs.Edit(PixelLayout control)
at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
at System.Windows.Threading.DispatcherOperation.InvokeImpl()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at MS.Internal.CulturePreservingExecutionContext.Run(CulturePreservingExecutionContext executionContext, ContextCallback callback, Object state)
at System.Windows.Threading.DispatcherOperation.Invoke()
at System.Windows.Threading.Dispatcher.ProcessQueue()
at System.Windows.Threading.Dispatcher.WndProcHook(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
at MS.Win32.HwndWrapper.WndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
at MS.Win32.HwndSubclass.DispatcherCallbackOperation(Object o)
at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
at System.Windows.Threading.Dispatcher.LegacyInvokeImpl(DispatcherPriority priority, TimeSpan timeout, Delegate method, Object args, Int32 numArgs)
at MS.Win32.HwndSubclass.SubclassWndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam)
at MS.Win32.UnsafeNativeMethods.DispatchMessage(MSG& msg)
at System.Windows.Threading.Dispatcher.PushFrameImpl(DispatcherFrame frame)
at System.Windows.Application.RunDispatcher(Object ignore)
at System.Windows.Application.RunInternal(Window window)
at MonoGame.Tools.Pipeline.Program.Main(String[] args)

Sebastian506563 · 2018-12-05T13:09:53Z

@rmhopkins I have the same issue, Have you find a solution?

rmhopkins · 2018-12-05T13:16:31Z

@Sebastian506563
I didn't find a solution, but when I created a new blank Monograme project in visual studio and added the reference to the content pipeline tool it succeeded, so I just copied all my code across to the new project. Not ideal, but it's got me back to being able to work again.

picnic · 2018-12-06T14:54:46Z

@rmhopkins Could you please provide a sample working project with XML import, please? I'm still struggling to get it to work. Even a very simple project would help me a lot.

picnic · 2018-12-07T15:19:36Z

Right now, I can get XML importer to work only with Monogame 3.6. So, each time I modify an XML file, I have to install Monogame 3.6 in order to generate the XNB files and then install the last version of Monogame to start the game. I could stay on Monogame 3.6 but there is a bug with the fonts.

harry-cpp added this to the 3.7.1 Release milestone Sep 28, 2018

harry-cpp added a commit to harry-cpp/MonoGame that referenced this issue Sep 28, 2018

[Pipeline Tool] Fix dependency loading (fixes MonoGame#6443)

213ba6d

Jjagg closed this as completed in 4a31ded Sep 29, 2018

nkast mentioned this issue Apr 26, 2019

Animated Model Processor is Nonfunctioning in MG 3.7.1 tainicom/Aether.Extras#10

Closed

QuinnBast mentioned this issue Oct 22, 2019

Content Pipeline Tool crashes when adding an assembly that references another assembly #6676

Closed

QuinnBast mentioned this issue Nov 14, 2019

Pipeline crashes when adding references #6942

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MonoGame Pipeline Tool Custom Processor bug #6443

MonoGame Pipeline Tool Custom Processor bug #6443

Quinn-L commented Sep 26, 2018 •

edited

harry-cpp commented Sep 28, 2018

harry-cpp commented Sep 28, 2018

Quinn-L commented Sep 29, 2018

Jjagg commented Sep 29, 2018

harry-cpp commented Sep 29, 2018

harry-cpp commented Sep 29, 2018

Jjagg commented Sep 29, 2018

harry-cpp commented Sep 29, 2018

picnic commented Oct 14, 2018 •

edited

harry-cpp commented Oct 14, 2018

rmhopkins commented Dec 4, 2018

Sebastian506563 commented Dec 5, 2018

rmhopkins commented Dec 5, 2018 •

edited

picnic commented Dec 6, 2018

picnic commented Dec 7, 2018

MonoGame Pipeline Tool Custom Processor bug #6443

MonoGame Pipeline Tool Custom Processor bug #6443

Comments

Quinn-L commented Sep 26, 2018 • edited

What version of MonoGame does the bug occur on:

What operating system are you using:

harry-cpp commented Sep 28, 2018

harry-cpp commented Sep 28, 2018

Quinn-L commented Sep 29, 2018

Jjagg commented Sep 29, 2018

harry-cpp commented Sep 29, 2018

harry-cpp commented Sep 29, 2018

Jjagg commented Sep 29, 2018

harry-cpp commented Sep 29, 2018

picnic commented Oct 14, 2018 • edited

harry-cpp commented Oct 14, 2018

rmhopkins commented Dec 4, 2018

Sebastian506563 commented Dec 5, 2018

rmhopkins commented Dec 5, 2018 • edited

picnic commented Dec 6, 2018

picnic commented Dec 7, 2018

Quinn-L commented Sep 26, 2018 •

edited

picnic commented Oct 14, 2018 •

edited

rmhopkins commented Dec 5, 2018 •

edited