Support for Azure projects #5

Closed
sayedihashimi opened this Issue Apr 27, 2012 · 58 comments

Projects

None yet
@sayedihashimi
Owner

We should support transforms for Azure projects.

References:
http://www.olegsych.com/2010/12/config-file-transformation/

@davidpeden3

Hey Sayed, I'd really love to see this feature implemented. Thanks!

@sayedihashimi
Owner

To be more specific, the feature you are looking for is for app.config files to be transformed when publishing an Azure Worker Role? If so I can give you a new build in a day or two if you are willing to try it out.

@davidpeden3

Hey Sayed,

Amazing.... I was literally typing an email to you asking this VERY thing after just having read this post and your response:

I’m absolutely interested in trying it out. Please do!

Thanks,

David

-----Original Message-----
From: Sayed Ibrahim Hashimi [mailto:reply@reply.github.com]
Sent: Wednesday, May 23, 2012 1:58 AM
To: davidpeden3
Subject: Re: [slow-cheetah] Support for Azure projects (#5)

To be more specific, the feature you are looking for is for app.config files to be transformed when publishing an Azure Worker Role? If so I can give you a new build in a day or two if you are willing to try it out.


Reply to this email directly or view it on GitHub:

#5 (comment) #5 (comment)

@sayedihashimi
Owner

FYI I thought this was going to be pretty simple but it looks like it is a bit more complicated. I'm going to have to understand the build process of Cloud Projects a bit better. Will have to investigate this further. I'll keep you posted, it is the top item on my priority list for this project.

@davidpeden3

Thanks for the update, Sayed. Yeah, I couldn't figure out how to get it to work myself. I have a build target specified in my Azure project to copy the file and it seems to be ignored despite the fact that the file is successfully transformed by Slow Cheetah. Keep me posted, I'm very interested in getting this working.

Also, I noticed that the "Preview Transform" bypasses my configured diff tool. I would really prefer that it use the tool that is configured directly in Visual Studio. I'm using Visual Studio 10.0.40219.1 SP1Rel.

Thanks!

-----Original Message-----
From: Sayed Ibrahim Hashimi [mailto:reply@reply.github.com]
Sent: Thursday, May 24, 2012 1:02 AM
To: davidpeden3
Subject: Re: [slow-cheetah] Support for Azure projects (#5)

FYI I thought this was going to be pretty simple but it looks like it is a bit more complicated. I'm going to have to understand the build process of Cloud Projects a bit better. Will have to investigate this further. I'll keep you posted, it is the top item on my priority list for this project.


Reply to this email directly or view it on GitHub:
#5 (comment)

@davidpeden3

Hey Sayed, I figured out how to do it. First I set MSBuild's verbosity to "Detailed" (I normally run with "Quiet"). This gave me the clue that I needed.

After doing so, I reviewed the build output and noticed the following:

    Target "CopyWorkerRoleFiles" in file "C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Windows Azure Tools\1.6\Microsoft.WindowsAzure.targets" from project "C:\hourstrackerweb\Source\Peden.HoursTracker.Azure\Peden.HoursTracker.Azure.ccproj" (target "ResolveSingleWorkerRoleReference" depends on it):
        Task "Copy"
            ...
            Copying file from "C:\hourstrackerweb\Source\Peden.HoursTracker.Worker.MailSender\bin\Peden.HoursTracker.Worker.MailSender.dll.config" to "C:\hourstrackerweb\Source\Peden.HoursTracker.Azure\obj\Release\HoursTracker.MailSender\Peden.HoursTracker.Worker.MailSender.dll.config".
            Copying file from "C:\hourstrackerweb\Source\Peden.HoursTracker.Worker.MailSender\app.config"                                          to "C:\hourstrackerweb\Source\Peden.HoursTracker.Azure\obj\Release\HoursTracker.MailSender\Peden.HoursTracker.Worker.MailSender.dll.config".
            ...

I've ommitted a bunch of output with the "..." and shown only the relevant part. You'll notice that the transformed file is being overwritten by "app.config". So what I did was created a build target in my Azure project like this:

<Target Name="CopyWorkerRoleConfigurations" AfterTargets="CopyWorkerRoleFiles" Condition="'$(Configuration)' == 'Release'">
    <Copy SourceFiles="..\Peden.HoursTracker.Worker.MailSender\bin\Peden.HoursTracker.Worker.MailSender.dll.config" DestinationFolder="$(IntermediateOutputPath)HoursTracker.MailSender" OverwriteReadOnlyFiles="true" />
</Target>

Which immediately follows the above task with output like this:

    Target "CopyWorkerRoleConfigurations" in project "C:\hourstrackerweb\Source\Peden.HoursTracker.Azure\Peden.HoursTracker.Azure.ccproj" (target "ResolveSingleWorkerRoleReference" depends on it):
        Task "Copy"
            Copying file from "..\Peden.HoursTracker.Worker.MailSender\bin\Peden.HoursTracker.Worker.MailSender.dll.config" to "obj\Release\HoursTracker.MailSender\Peden.HoursTracker.Worker.MailSender.dll.config".

Thus restoring my transformed config file. It's not a perfect solution but it definitely works. Hope that helps.

@1kevgriff

I'm also looking for this feature/fix to be implemented. I'm definitely willing to try out any new builds.

@sayedihashimi
Owner

Thanks guys I will take a look at this later today and get back to you. Once I have a new build I will share it with you all. Thanks for your willingness to help, I appreciate it.

@sayedihashimi
Owner

Hey guys I looked at this and I might have something which works. I've uploaded the vsix to http://dl.dropbox.com/u/40134810/SlowCheetah-Azure-AppConfig-Fix.zip. Can you un-install the current verison and then install this on top of it? After you open a solution and build the file at C:\Users{USER-NAME}\AppData\Local\Microsoft\MSBuild\SlowCheetah\v1\SlowCheetah.Transforms.targets should get updated. I have not tried a full Azure publish but the config file in the obj file is correct. If you guys confirm that it works I will need to verify that it doesn't break any existing scenarios.

@centur
centur commented May 28, 2012

I implemented config transformation (with some conventions) by hands. Beware - there is multiple issues when Web Role have more that one website in it: no explicit "build" target called for second and followed sites. Just file copying. Maybe new incoming Azure will shed some light on this issues with Azure build tasks, but current is just a mess - hard to automate with msbuild :(

@1kevgriff

Your drop works fine for packaging. But I was still having issues with local emulation. It seems that after transformation, CSPack copies over the original app.config (non-transformed). I was able to hack around it by doing this in my cloud project (similar to what @davidpeden3 did):

<Target Name="CopyWorkerRoleConfigurations" AfterTargets="CopyWorkerRoleFiles">
    <Copy SourceFiles="..\Winsitter.Worker\bin\$(Configuration)\Winsitter.Worker.dll.config"
          DestinationFolder="$(IntermediateOutputPath)Winsitter.Worker" OverwriteReadOnlyFiles="true" />
</Target>

I'm not sure why it work for packaging though, but not for local emulation.

@sayedihashimi
Owner

OK thanks for letting me know it doesn't work for the emulator. The problem with the approach to extend CopyWorkerRoleFiles is that SlowCheetah needs a mechanism to do all this from the client project, i.e. it cannot modify the behavior of the Azure project. I will look into this and get back to you all.

@sayedihashimi
Owner

@1kevgriff are you sure that it's not working? Based on what I'm seeing it does work. Can you close all instances of VS, reopen the solution, clean and try again?
I created a sample worker role which writes out the app settings to a file, you can grab my sample at: http://dl.dropbox.com/u/40134810/AzureWorker.zip to try it out.

FYI I have Azure SDK 1.6 installed, its the latest version. Is that what you have?

@oyen
oyen commented May 30, 2012

Hi, I installed (http://dl.dropbox.com/u/40134810/SlowCheetah-Azure-AppConfig-Fix.zip) but that didn't work when I packaged and Upgraded my Azure project -- the Worker Role's app.Staging.config settings were not applied.

What happened was, after the Package was created, I checked the my /bin/Staging/WorkerRoleProject.dll.config file (am I looking at the right file?) and saw that the correct settings were shown. But those settings were not reflected on the Azure project. I still had to change the main app.config file manually for everything to work.

@1kevgriff

@sayedihashimi Your demo worked for me. Curious why it didn't work in my project.

@sayedihashimi
Owner

@1kevgriff and @oyen is your project using Azure SDK 1.6? Can you try to do a clean and see if that helps?

@oyen
oyen commented May 31, 2012

@sayedihashimi

  1. I tried the "..." elements on .ccproj and that solved my issue.
  2. I have: Windows Azure Tools for Microsoft Visual Studio 2010 - November 2011 v1.6.41103.1601
    is that what you meant?
  3. By clean, you mean clean the solution then do another package - but without the "<Target...>" elements on .ccproj?
@sayedihashimi
Owner

Hi @oyen what do you mean by #1? I'm not that familiar with worker roles so pardon my ignorance.

Yes by clean I meant to clean the solution then try to package/publish/emulate.

@oyen
oyen commented May 31, 2012

@sayedihashimi
Sorry that was a typo, I meant TARGET elements. #1 was just a confirmation that @1kevgriff 's solution works.

Cleaning the solution doesn't help :)

Thank you for replying actively.

@sayedihashimi
Owner

OK thanks, can anyone provide a sample project which it doesn't work so that I can trouble shoot it?

If you can't give a sample can you send build logs? A detailed and diagnostic log would be ideal. You can use the following options to msbuild.exe to get that /flp1:v=d;m.d.log /flp2:v=diag;m.diag.log.

@davidjrh

Hi Sayed,

just downloaded the latest bits available on the Visual Studio gallery, and I can confirm that the transformations are not taking effect on packaging. Are these fixes included in the actual available release?

P.S.I'm using the latest SDK 1.7 that was released jun-07

@davidjrh

BTW, I've downloaded your test Project from Dropbox, and after opening (and converting to the latest SDK 1.7), and after packaging the same problem appears.

@davidjrh

Hi @michaelpimentel, AFIK, the addin only processes worker roles config files (like app.config), not web roles, where the VS transformations do the work.

@sayedihashimi
Owner

@davidjrh is correct for web projects the web.config is not transformed during publishing but only during publishing/packaging.

@davidjrh Perhaps there was a breaking change in Azure SDK 1.7 for the feature in 1.6 which I was relying. I am traveling right now so it's not easy for me to work on this but once I get back I will take a look at this. Thanks a lot for your help, I appreciate it.

@aarondcoleman

+1 Trying to do this with Azure 1.7 SDK. Also, the link to the DropBox now points to a missing file. Would love a solution to this. Thanks.

@jstawski

@sayedihashimi, your work is greatly appreciated. Just writing here that I am also having the same problem with worker roles and packaging. I have downloaded the latest VSIX using the Extension Manager of Visual Studio 2010. To be more precised I'm using version 2.3.

@jstawski

After posting the comment above I decided to see how I can help on this project and decided to contribute at least a little bit of my personal experience. Following @davidpeden3 post I was able to copy the transformed app.config into the built package using the 1.7 SDK. I confirmed that it worked for both my worker role and my web role for both debug and release. All I did was add the following Target as the last node before the :

<Target Name="CopyWorkerRoleConfigurations" AfterTargets="CopyWorkerRoleFiles">
    <Copy SourceFiles="..\MyProject\$(OutDir)MyProject.dll.config" 
DestinationFolder="$(IntermediateOutputPath)MyProject" OverwriteReadOnlyFiles="true" />
</Target>

If you accept outside contribution I could try and help implement this.

@sayedihashimi
Owner

Hi @jstawski I would love help implementing this and of course I accept outside contributions :) I haven't had a chance to revisit this lately but may have some cycles towards the end of the week. If you get to it before I do all the changes are contained into its own branch https://github.com/sayedihashimi/slow-cheetah/tree/azureSupport.

@davidpeden3

I decided to revisit this topic to see if I could make my solution generic. It occurred to me that most of the information necessary is in the azure project. Using target/task batching, you can loop through all of the roles like this:

<Target Name="PrintRoles" Outputs="%(ProjectReference.Name)">
    <Message Text="Include: @(ProjectReference)" />
    <Message Text="Name: @(ProjectReference->'%(Name)')" />
    <Message Text="Project: @(ProjectReference->'%(Project)')" />
    <Message Text="Private: @(ProjectReference->'%(Private)')" />
    <Message Text="RoleType: @(ProjectReference->'%(RoleType)')" />
    <Message Text="RoleName: @(ProjectReference->'%(RoleName)')" />
</Target>

Unfortunately, each ProjectReference does not contain the $(AssemblyName) property from the referenced project. For me, this is a deal breaker because I need this to derive the config file name and there is no guarantee that the AssemblyName matches the Project or RoleName. At first, I figured that you could dynamically import the project file inside of the target but it turns out that MSBuild doesn't support Import inside of a Target.

Sayed, would it be possible to get the Azure team to update their project file to include this key information? With this one property, it would be totally possible to copy files for each worker role without having to remember to come in and edit the Azure project every time a role was added/removed. The target would look like this:

<Target Name="CopyWorkerRoleConfigurations" Outputs="%(ProjectReference.Name)" AfterTargets="CopyWorkerRoleFiles">
    <PropertyGroup>
        <ProjectPath>@(ProjectReference)</ProjectPath>
        <ProjectName>@(ProjectReference->'%(Name)')</ProjectName>
        <ProjectFolder>$([System.String]::Copy($(ProjectPath)).Replace("$(ProjectName).csproj",''))</ProjectFolder>
        <!-- This is the missing AssemblyName per role.  Pretend it was here... -->
        <ConfigAssemblyName>WishThisWereHere.dll</ConfigAssemblyName>
        <ConfigFilePath>$(ProjectFolder)$(OutDir)$(ConfigAssemblyName).config</ConfigFilePath>
    </PropertyGroup>
    <Copy SourceFiles="$(ConfigFilePath)" DestinationFolder="$(IntermediateOutputPath)$(ProjectName)" OverwriteReadOnlyFiles="true" Condition="@(ProjectReference->'%(RoleType)') == 'Worker'" />
</Target>

Alternatively, do you know of a workaround for this approach?

@bojingo
bojingo commented Nov 13, 2012

Have there been any new developments with this issue? Would really love to have this working out of the box with this extension!

@sayedihashimi
Owner

There are no developments here. I spent several hours a few months ago on this. I thought I had something. working, but evidently not.
Unfortunately I don't think I have the knowledge of the build/publish process for Cloud Services to complete this. If someone out there who does know would like to chip in here I'd love to help.

@davidpeden3

Hey Sayed,

Were you able to take a look at my last post and work I did? We’re just missing the one key attribute…

@sayedihashimi
Owner

Thanks guys the difference here is that the solution needs to be implemented such that the project containing the worker role code is modified, not the Azure Service Project.

With that said I think the info here may help me solve the problem. I am trying to carve out sometime to look at this again. Sorry for the delay here everyone, I've been slammed with requests in different directions on this project and having a hard time prioritizing the items that I'm working on.

@davidpeden3

Hey Sayed, thanks for the update. Might be worth making a UserVoice site to solicit general public prioritization. ;)

@sayedihashimi
Owner

Thanks David, I think that's a great idea. I have setup a uservoice at http://slowcheetah.uservoice.com. I created the first suggestion for this item at http://slowcheetah.uservoice.com/forums/185106-general/suggestions/3385300-support-for-azure-worker-roles.

Going forward I'll just be working on all the most voted items.

@sayedihashimi
Owner

OK I think I have something working for app.config transforms in Worker Roles. I've tested with Azure SDK 1.8 and have placed the latest build at https://dl.dropbox.com/u/40134810/SlowCheetah/issue-5/SlowCheetah-issue5.vsix. Could you guys try it out and let me know if it works?

I'm not sure if the current support will cover files other than app.config so if you do notice please let us know! :)

@centur
centur commented Dec 4, 2012

Web.config transformed, at least for "single web site in webrole" setup.
App.config for worker role NOT transformed.

@brandonpowers

I just tested SlowCheetah-issue5.vsix. The App config created in the bin/release folder was transformed correctly when building the solution normally for azure worker roles. It was not transformed correctly when right clicking the azure cloud project and selecting "package" for deployment. This was determined by opening the *.Cloud.cspkg file in 7zip and locating the app config for the Azure Worker role project.

@sayedihashimi
Owner

@brandonpowers do you have Azure SDK 1.8? I thought this was working in the package case for 1.8.

@brandonpowers

Ah, looks like I'm on Azure SDK 1.4. I'll update and try again. Thanks.

@davidpeden3

Hey Sayed,

I downloaded the issue 5 patch and removed my CopyWorkerRoleConfigurations target and the worker role was properly transformed and included in the package. Everything looks good from my perspective. Hopefully others will chime in as well so we can mark this issue closed. Thanks for sticking with it!

@sayedihashimi
Owner

Thanks @davidpeden3. Since you've verified it I have merged the code from the azure branch and into master. I've updated the drop at https://dl.dropbox.com/u/40134810/SlowCheetah/issue-5/SlowCheetah-issue5.vsix.

I have noticed that if you have a transform file, which is not for app.config, in the worker role the Azure package will fail to build. I am investigating that.

@vkam
vkam commented Dec 20, 2012

I need to have the app.config transforms for the Azure worker role and a note that we are using TFS preview builds. I tried the slow cheetah a couple of hours ago and I was able to publish and deploy, but the configs were not transformed. So I came across your new drop an hour ago at https://dl.dropbox.com/u/40134810/SlowCheetah/issue-5/SlowCheetah-issue5.vsix and I installed it. Now when I run the worker role locally with debug, release configs, I am able to see the transforms. I then added a new transform called mytest and it works too. I then tried to publish it using the TFS preview's build definition as I have been doing, but this time I get an error and can't even deploy. Not sure if the new drop broke something in the tfs build process?

"Exception Message: Invalid enum value 'SuspendedTransitioning' cannot be deserialized into type 'Microsoft.TeamFoundation.Deployment.Workflow.AzureManagement.v1_7.DeploymentStatus'. Ensure that the necessary enum values are present and are marked with EnumMemberAttribute attribute if the type has DataContractAttribute attribute. (type SerializationException)
Exception Stack Trace: at System.Activities.Statements.Throw.Execute(CodeActivityContext context)
at System.Activities.CodeActivity.InternalExecute(ActivityInstance instance, ActivityExecutor executor, BookmarkManager bookmarkManager)
at System.Activities.Runtime.ActivityExecutor.ExecuteActivityWorkItem.ExecuteBody(ActivityExecutor executor, BookmarkManager bookmarkManager, Location resultLocation)"

@sayedihashimi
Owner

@vkam I highly doubt that SlowCheetah had anything to do here. If you get an error from TFS preview during a build, if it was related to SlowCheeth you'd get an MSBuild error. This looks like an error on the TFS side. If you are able to reproduce it please open a new issue, and include the specific steps required.

FYI If you are looking to hook up SlowCheetah on a build server see my blog post at: http://sedodream.com/2011/12/12/SlowCheetahXMLTransformsFromACIServer.aspx

@vkam
vkam commented Dec 21, 2012

Today, I uninstalled and reinstalled your new drop (https://dl.dropbox.com/u/40134810/SlowCheetah/issue-5/SlowCheetah-issue5.vsix.). Created a new Azure worker role app, created config transforms and deployed to cloud using TFS online's bulild definition. No TFS errors this time, but configs are still not transformed although all transforms work beautifully locally on F5. Yesterday, I got the tfs error multiple times, so not sure what changed. I will take a look at the link you provided and will try to see if I can make it work.

@vkam
vkam commented Dec 21, 2012

Okay, I started reading the blog post you suggested http://sedodream.com/2011/12/12/SlowCheetahXMLTransformsFromACIServer.aspx, but first thing I notice is that i don't see any slow cheetah related properties or imports added to the ccproj file. Am I missing something here?

@vkam
vkam commented Dec 21, 2012

An updae to my previous post: I created a simple C# console app (a non Azure) and added app.config transforms and I can see that slowcheetah properties are added to the project file, but on a Azure worker role sample project, no slowcheetah related elements are added.

@sayedihashimi
Owner

@vkam there are no changes to the Azure project required, only need to modify the actual project of the worker role.

@vkam
vkam commented Dec 21, 2012

Okay, I see that properties are included in the workerrole project. I then did exactly what you described in http://sedodream.com/2011/12/12/SlowCheetahXMLTransformsFromACIServer.aspx. But now the build fails.

"C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v11.0\Windows Azure Tools\1.8\Microsoft.WindowsAzure.targets (2713): Could not copy the file "C:\a\bin\obj\test\SlowCheetah\app.config" because it was not found" . The "test" is my app.config transform file (app.test.config).

Where's the folder C:\a\bin\ coming from? Is it where Slowcheetah is trying to create a new copy of transformed app.config? BTW, I am using "TFS online", so we don't know which folder it's doing the build on?

@sayedihashimi
Owner

@vkam please open a new issue for this, as it is not related to this specific feature.

@sayedihashimi
Owner

As I noted previously if you have a project where a file other than app.config is being transformed you will get a build error if its a worker role. I've opened #61 to track this. I'll make sure to address that before this functionality is released.

@sayedihashimi
Owner

In my previous comment I mentioned that I opened a new bug for the non-app.config file. After opening I realized that it was a duplicate of #44. I've closed #61 because of that.

Good news I think I have the bug fixed. So for Azure Worker Roles you should be able to get app.config, as well as other XML files updated.

If you are interested in followup lets discuss at #44 if it is related to the non-app.config file support.

You can get the build with the fix at https://dl.dropbox.com/u/40134810/SlowCheetah/issue-44/SlowCheetah-issue-44.zip. I have removed the link to the previous build from this thread.

If you have a minute I would really appreciate if you guys could try this out.

_As soon as you all sign off I will publish the update_

@apwestgarth

I've tried the update with solution wich has a webapi role with web.config transforms and a worker role with app.config transforms and can confirm that all transforms have been applied on package of my cloud solution :)

@gautambjain

Yes, I am having a cloud project with WebRole & WorkerRole and I can confirm that app.config is getting transformed when the "Package" feature is used on the Cloud project.

@tommck
tommck commented Jan 28, 2013

Any news on when this will make it into the Visual Studio Gallery? Many people won't be excited about downloading things a different way.

@sayedihashimi
Owner

I'm almost there. I had a bug #44 that I needed to fix before releasing.

I have just fixed that and have a new drop available at https://dl.dropbox.com/u/40134810/SlowCheetah/issue-44/SlowCheetah-issue-44-v2.zip. Could you guys try it out and let me know if we are good to release this. In this build both app.config as well as other files should be transformed.

Once we have two people verify this I will release it, there are no other blockers here.

@tommck
tommck commented Jan 30, 2013

I'll try that out for the app.config transforms. I'm not doing the other types (yet), so I won't be sure. The previous version of the fix did work, however.

@tommck
tommck commented Jan 30, 2013

I just tried it and the app.config transforms in my Worker Role worked properly.

Thanks for all your work, Sayed!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment